Predictive model for assisted differential diagnosis

[rubanzasilva]

Project Name

Predictive model for assisted differential diagnosis

Description

A predictive model that takes in a patient's complaints and returns a differential diagnosis. The idea is for such tools to be used alongside doctors to help ease their work in diagnosis diseases accurately.

Added future functionality would be to increase the scope of the input to take in medical history data etc.

Build

Yes

Train

Yes

Analyze

No

Challenge Topic / Topic Category

• Customized Information Extraction
• Education
• Enhancing Accessibility in Healthcare Through AI
• Mitigating Natural Disasters in a Changing Climate
• Reducing Food Waste Through AI-Powered Innovation
• other - World Issues
• other - Open-Source
• other - Scientific Computing
• other

Project Repository URL

https://github.com/rubanzasilva/symptom_to_disease

Deployed Endpoint URL

No response

Project Video File (not folder) Link (ensure viewer access)

https://drive.google.com/file/d/1aHhjgn6oSx37YQDwsKUuXdGnHupnuK4r/view?usp=sharing

Team Members

@rubanzasilva

[naumnaum]

Your project tackles a crucial area in healthcare by aiming to assist doctors in accurate diagnosis, which can significantly enhance the efficiency of the diagnostic process. Training an LSTM model on a specific dataset of diseases is a solid starting point for processing natural language input effectively.

That said, LSTM, while foundational, is becoming an outdated architecture for tasks like this. Leveraging modern LLMs coupled with rigorous validation of their responses could be a more robust and scalable solution.

Here’s a potential approach:

1. Implement a Retrieval-Augmented Generation (RAG) pipeline using keyword-based or vector search to identify relevant data in your dataset.

2. Utilize an LLM to validate and analyze the retrieved information.

3. Generate a human-readable, accurate response with the LLM, ensuring alignment with the data.

This approach could enhance both the accuracy and the interpretability of the model's outputs, making it a reliable tool for medical professionals.

Your mention of expanding the scope to include medical history data is an excellent future direction—it adds context to the predictions and aligns well with real-world diagnostic workflows.

This is a promising project with great potential to support healthcare professionals. Keep up the great work!

[rubanzasilva]

Your project tackles a crucial area in healthcare by aiming to assist doctors in accurate diagnosis, which can significantly enhance the efficiency of the diagnostic process. Training an LSTM model on a specific dataset of diseases is a solid starting point for processing natural language input effectively.

That said, LSTM, while foundational, is becoming an outdated architecture for tasks like this. Leveraging modern LLMs coupled with rigorous validation of their responses could be a more robust and scalable solution.

Here’s a potential approach:

1. Implement a Retrieval-Augmented Generation (RAG) pipeline using keyword-based or vector search to identify relevant data in your dataset.
2. Utilize an LLM to validate and analyze the retrieved information.
3. Generate a human-readable, accurate response with the LLM, ensuring alignment with the data.

This approach could enhance both the accuracy and the interpretability of the model's outputs, making it a reliable tool for medical professionals.

Your mention of expanding the scope to include medical history data is an excellent future direction—it adds context to the predictions and aligns well with real-world diagnostic workflows.

This is a promising project with great potential to support healthcare professionals. Keep up the great work!

@naumnaum I appreciate you taking the time to look at my work and to provide feedback.

Originally, I planned to add another method that I understand uses the same basis as modern large language models.
The approach would be the ULMFiT approach described here. Due to the time constraints on my end, I never got to do this.

I will try this approach and upload it to a new branch to keep the work on the main branch within the hackathon time limits.

But even the implementation I had in mind was just using a small language model of sorts, having seen the approach you have suggested, I will also try implementing it using the RAG pipeline approach you suggested and further fine-tuning based on the steps you suggested.
Thank you for your insight.

Finally, I will compare both method's performance on unseen data, etc.