MNLobago/medgemma3-multimodal-health001

Med-Gemma 4B-IT LoRA Adapter

This repository contains a PEFT LoRA adapter fine-tuned on google/medgemma-4b-it for health misinformation and reliability classification on paired text-and-image content.

Model Summary

• —Base model: google/medgemma-4b-it
• —Adapter type: LoRA / PEFT
• —Task: Binary classification (reliable / unreliable)
• —Modality: Text + image
• —Training setup: QLoRA-style 4-bit NF4 with BF16 compute
• —Primary use: Detect cross-modal inconsistencies in pandemic and medical content

What This Model Is For

Use this adapter to flag health content where the text and image disagree, or where the content is otherwise unreliable. It is intended for research, benchmarking, and human-in-the-loop screening, not for clinical decision-making.

Key Results

• —Med-Gemma: 100% on the WHO-curated 28-case adversarial suite
• —Gemma 3: 82% on the same suite
• —Text-only BERT notebook: 61% on the same 28-case stress suite

Important: the 100% result is from the WHO-curated adversarial suite.

Training Details

• —LoRA rank: 16
• —LoRA alpha: 32
• —LoRA dropout: 0.05
• —Target modules: q_proj, k_proj, v_proj, o_proj
• —Learning rate: 1e-4
• —Epochs: 5
• —Batch size per GPU: 4
• —Gradient accumulation: 8
• —Warmup ratio: 0.05
• —Optimizer: AdamW
• —Scheduler: cosine annealing
• —Early stopping patience: 2

The multimodal Gemma and Med-Gemma runs were trained on TPU for 5 epochs. That schedule was chosen because a single epoch took about 1 hour 30 minutes on the full dataset, and the dataset could be explored as image-level and text-level subsets with different observations from the EDA. The BERT baseline was trained on GPU for 10 epochs in the text-only notebook. Future work can extend the same pipeline with longer training runs to see whether additional epochs improve robustness.

See config/training_config.yaml and notebooks/02_training/healthcare-multimodal-tpu.ipynb for the exact training pipeline.

Evaluation Notes

The repo reports the WHO-curated 28-case adversarial suite, which is used to compare Med-Gemma, Gemma 3, and the text-only BERT notebook fairly.

The text-only BERT notebook reports 61% on the same 28-case stress suite, while its in-distribution result remains 95.4% on the standard test set.

Intended Use and Limitations

Recommended use is limited to research and screening workflows where model outputs are reviewed by a human. Do not use the model as a diagnostic tool or as a standalone medical authority.

Known limitations include dataset bias, domain shift outside the biomedical and pandemic setting, and reduced reliability on ambiguous or low-quality inputs.

How to Run

Load the base model, attach the adapter with PEFT, and use the paired processor saved alongside the adapter. See the training and inference notebooks in this repository for the exact setup.

Citation

@article{lobago2025medlogic,
  title={Benchmarking Multimodal Misinformation Detection in Pandemic Health Content},
  author={Lobago, Moan and Tuyambaze, Yvette and Niyogisubizo, Johovanis},
  year={2025},
  note={MedLogic Gap Detector project}
}

Repository References

• —GitHub repository: https://github.com/MNLobago/MedLogic-Gap-Detector/tree/main
• —Training notebook: notebooks/02_training/healthcare-multimodal-tpu.ipynb
• —Text-only comparison notebook: justtextINF.ipynb
• —Evaluation summary: docs/RESULTS.md
• —Methodology: docs/METHODOLOGY.md