Modelpublic

BennyZhanghaoChan/Task3-2D-LIST-SEU-Zhanghao_Chen-Testing_Submission

sourceHugging Faceapache-2.0updated 11d agoView on Hugging Face

1likes

Model Card

Two-Stage Mixture-of-LoRA for Multi-Task Medical Vision-Language Learning

This repository contains the model adapter released for the paper Two-Stage Mixture-of-LoRA for Multi-Task Medical Vision-Language Learning.

Code: https://github.com/YuanYL03/MICCAI-FLARE-2026-Challenge-Task3-2D

The model is built on MedGemma-1.5-4B with a shared-specific Mixture-of-LoRA architecture comprising one shared LoRA and six task-specific expert LoRAs, and supports six benchmark tasks: disease diagnosis classification, multi-label classification, detection, cell counting, regression, and report generation.

Quick Start

1. Clone and create an environment

bash

git clone https://github.com/YuanYL03/MICCAI-FLARE-2026-Challenge-Task3-2D.git
cd MICCAI-FLARE-2026-Challenge-Task3-2D

conda create -n flare26-medgemma python=3.11 -y
conda activate flare26-medgemma

# Install a CUDA-compatible PyTorch build first; choose the command for your CUDA version from pytorch.org.
pip install torch torchvision
pip install -r inference/requirements.txt

2. Download the base model and adapter bundle

bash

# Base model (requires accepted MedGemma access terms)
huggingface-cli download google/medgemma-1.5-4b-it \
  --local-dir checkpoints/medgemma-1.5-4b-it

# Shared adapter, six task experts, tokenizer, and processor files
huggingface-cli download BennyZhanghaoChan/Task3-2D-LIST-SEU-Zhanghao_Chen-Testing_Submission \
  --local-dir checkpoints/medgemma-best-0828

3. Run inference on raw FLARE data

bash

bash inference/run_inference.sh \
  --data_root /path/to/FLARE_Test \
  --model_base checkpoints/medgemma-1.5-4b-it \
  --adapter_path checkpoints/medgemma-best-0828 \
  --output_dir outputs/testing \
  --split_name testing \
  --device cuda \
  --image_size 896 \
  --max_images 1 \
  --max_new_tokens 256 \
  --temperature 0