CoolFace
Modelpublic

KYAGABA/testmodel

sourceHugging Faceotherupdated 4mo agoView on Hugging Face
0likes7downloads
Model Card

MedGemma 27B - Maternal, Sexual & Reproductive Health Oracle for African Languages

Fine-tuned Google MedGemma 27B Text for the Zindi ITU Multilingual Health QA Challenge.

Specialized in answering Maternal, Sexual, and Reproductive Health (MSRH) questions in:

  • —Akan (Twi/Fante from Ghana)
  • —Amharic (Ethiopia)
  • —Luganda (Uganda)
  • —Swahili (Kenya)
  • —English (Ethiopia, Ghana, Kenya, Uganda)

Model Description

LoRA adapter for google/medgemma-27b-text-it, fine-tuned on 29,815 multilingual medical Q&A samples across 8 language-region pairs.

Training Details

  • —Base model: google/medgemma-27b-text-it (27B params, medical text-only)
  • —Training method: QLoRA (4-bit quantization + LoRA)
  • —LoRA config: r=8, alpha=16, attention-only modules
  • —Trainable params: 16.7M (0.21% of total)
  • —Training data: 29,815 multilingual medical Q&A samples
  • —Optimizer: AdamW fused, lr=3e-5, linear warmup 5%
  • —Hardware: NVIDIA A40 (48GB VRAM)
  • —Final eval_loss: 1.39

Loss Trajectory

Stepeval_loss
6001.69
9001.58
12001.50
15001.45
18001.42
18641.39 (best)

Usage

python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel

quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

base_model = AutoModelForCausalLM.from_pretrained(
    "google/medgemma-27b-text-it",
    device_map="auto",
    torch_dtype=torch.bfloat16,
    attn_implementation="eager",
    quantization_config=quantization_config,
)

model = PeftModel.from_pretrained(base_model, "KYAGABA/medgemma-27b-msrh-african-oracle")
model.eval()

tokenizer = AutoTokenizer.from_pretrained("KYAGABA/medgemma-27b-msrh-african-oracle")

# Example
question = "How can young people access reproductive health services?"
language = "English"

prompt_text = f"Answer this question in {language} about maternal, sexual, and reproductive health: {question}"
messages = [{"role": "user", "content": prompt_text}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors='pt').to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=400,
        do_sample=False,
        num_beams=3,
        repetition_penalty=1.1,
    )

response = tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True)
print(response)

Dataset

Trained on the Zindi ITU Multilingual Health QA Challenge dataset:

SubsetSamplesLanguageRegion
Eng_Uga7,624EnglishUganda
Aka_Gha4,455AkanGhana
Eng_Gha4,443EnglishGhana
Eng_Eth3,915EnglishEthiopia
Lug_Uga3,383LugandaUganda
Eng_Ken2,080EnglishKenya
Swa_Ken2,070SwahiliKenya
Amh_Eth1,845AmharicEthiopia

Intended Use

For research and educational purposes to support healthcare information access in African languages. NOT for direct clinical use. Always consult qualified healthcare professionals.

Limitations

  • —May add English preamble at start of responses
  • —Lower quality for Akan compared to English (less training data)
  • —Trained for ~1.13 epochs only (compute constraints)
  • —Best for MSRH topics

Citation

@misc{medgemma27b-msrh-africa,
  author = {KYAGABA, Arul},
  title = {MedGemma 27B - MSRH African Oracle},
  year = {2026},
  publisher = {HuggingFace},
  howpublished = {https://huggingface.co/KYAGABA/medgemma-27b-msrh-african-oracle}
}

Acknowledgements

  • —Google for MedGemma 27B base model
  • —Zindi and ITU for the multilingual health QA challenge
  • —AfriMed-QA community for advancing African medical AI