blackshadow1/Multi-Modal-Medical-Analysis-System

MediSync: Multi-Modal Medical Analysis System

MediSync is an AI-powered healthcare solution that combines X-ray image analysis with patient report text processing to provide comprehensive medical insights.

Features

• —X-ray Image Analysis: Detects abnormalities in chest X-rays using pre-trained vision models from Hugging Face.
• —Medical Report Processing: Extracts key information from patient reports using NLP models.
• —Multi-modal Integration: Combines insights from both image and text data for more accurate diagnosis suggestions.
• —User-friendly Interface: Simple web interface for uploading images and reports.

Project Structure

mediSync/
├── app.py                    # Main application with Gradio interface
├── models/
│   ├── image_analyzer.py     # X-ray image analysis module
│   ├── text_analyzer.py      # Medical report text analysis module
│   └── multimodal_fusion.py  # Fusion of image and text insights
├── utils/
│   ├── preprocessing.py      # Data preprocessing utilities
│   └── visualization.py      # Result visualization utilities
├── data/
│   └── sample/               # Sample data for testing
└── tests/                    # Unit tests

Setup Instructions

1. 1.Clone this repository:

git clone [repository-url]
cd MediSync

1. 1.Install dependencies:

pip install -r requirements.txt

1. 1.Run the application:

python app.py

1. 1.Access the web interface at http://localhost:7860

Models Used

• —X-ray Analysis: facebook/deit-base-patch16-224-medical-cxr
• —Medical Text Analysis: medicalai/ClinicalBERT
• —Additional Support Models: Medical question answering and entity recognition models

Use Cases

• —Preliminary screening of chest X-rays
• —Cross-validation of radiologist reports
• —Educational tool for medical students
• —Research tool for studying correlation between visual findings and written reports

Note

This system is designed as a support tool and should not replace professional medical diagnosis. Always consult with healthcare professionals for medical decisions.