CoolFace
Apppublic

pascal-maker/medicalaiapp

sourceHugging Faceupdated 1y agoView on Hugging Face
0likes
App README

Medical VLM with SAM-2 and CheXagent

A comprehensive medical imaging analysis tool that combines:

  • —Qwen-VLM for medical visual question answering
  • —SAM-2 (Segment Anything Model 2) for automatic medical image segmentation
  • —CheXagent for structured chest X-ray report generation

Features

  1. 1.Medical Q&A: Ask questions about medical images using the Qwen-VLM model
  2. 2.Automatic Masking: Segment medical images automatically using SAM-2
  3. 3.Structured Report Generation: Generate detailed chest X-ray reports using CheXagent
  4. 4.Visual Grounding: Locate specific findings in medical images

Setup

  1. 1.Clone the repository:
bash
git clone https://github.com/pascal-maker/medicalvlm.git
cd medicalvlm
  1. 1.Create and activate a virtual environment:
bash
python -m venv chexagent_env
source chexagent_env/bin/activate  # On Windows: chexagent_env\Scripts\activate
  1. 1.Install dependencies:
bash
pip install -r requirements.txt
  1. 1.Download required model checkpoints:
  2. 2.SAM-2 checkpoint: Place in checkpoints/sam2.1_hiera_large.pt
  3. 3.Other model weights will be downloaded automatically on first run

Usage

Run the Gradio interface:

bash
python app.py

The web interface will be available at http://localhost:7860

Requirements

  • —Python 3.8+
  • —PyTorch
  • —CUDA-compatible GPU (recommended)
  • —See requirements.txt for full list of dependencies

License

[Your chosen license]

Acknowledgments