j4rias/medvision-edge-v4-merged

MedVision Edge v4 — Chest X-ray Screening (Merged Model)

Fine-tuned Gemma 4 E4B-it (8B params) for automated chest X-ray pathology detection. Screens 5 conditions simultaneously with validated clinical accuracy, generates WHO-compliant treatment protocols, and outputs in 140+ languages natively.

This repo contains the full merged model in fp16 (16GB). Ready for direct inference with transformers — no Unsloth or PEFT required. For the LoRA adapter weights only (660MB), see j4rias/medvision-edge-v4.

Model Details

Model Description

MedVision Edge is an AI-powered chest X-ray screening system designed for underserved communities where 2.2 billion people lack access to medical imaging (WHO, 2023). A community health worker photographs a chest X-ray with any smartphone and receives:

1. 1.Pathology detection for 5 conditions screened simultaneously
2. 2.WHO IMCI clinical protocols with evidence-based treatment guidelines (deterministic, zero hallucination)
3. 3.Weight-based drug dosing from verified lookup tables
4. 4.Referral urgency assessment with color-coded triage
5. 5.Native language output in 140+ languages via Gemma 4's built-in multilingual capability

The model was fine-tuned using Unsloth QLoRA on ~23,000 training examples derived from the NIH ChestX-ray14 dataset (112,120 images, 30,805 patients), with oversampling and augmentation for rare pathologies. The LoRA adapter was then merged back into the base model to produce this standalone fp16 checkpoint.

• —Developed by: Joel Arias (@j4rias)
• —Model type: Vision-Language Model (merged fine-tune of Gemma 4 E4B-it)
• —Language(s): 140+ languages (Gemma 4 native multilingual)
• —License: Apache 2.0
• —Fine-tuned from: google/gemma-4-e4b-it (Google Gemma 4 E4B-it)

Model Sources

• —Repository: github.com/J4rias/medvision-edge
• —Demo: huggingface.co/spaces/j4rias/MedVision-Edge

Uses

Direct Use

Load directly with transformers for inference on chest X-ray images:

from transformers import AutoModelForImageTextToText, AutoProcessor
from PIL import Image
import torch

model_id = "j4rias/medvision-edge-v4-merged"

# Load merged model (no PEFT/Unsloth needed)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
)
model.to("cuda")
processor = AutoProcessor.from_pretrained(model_id)

# Prepare input
image = Image.open("chest_xray.jpg").convert("RGB")
messages = [
    {"role": "user", "content": [
        {"type": "image"},
        {"type": "text", "text": "Analyze this chest X-ray for: Pneumonia, Consolidation, Cardiomegaly, Pleural Effusion, Pulmonary Edema. For each: state YES or NO, then describe findings."},
    ]}
]

inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_dict=True, return_tensors="pt",
).to("cuda")
inputs["images"] = processor.image_processor(image, return_tensors="pt")["pixel_values"].to("cuda", dtype=torch.float16)

output = model.generate(**inputs, max_new_tokens=512, temperature=0.1)
print(processor.decode(output[0], skip_special_tokens=True))

Note: This merged model requires ~8GB VRAM in fp16. For 4-bit quantized inference with lower VRAM, use the LoRA adapter with Unsloth instead.

Downstream Use

• —HuggingFace Spaces / ZeroGPU: This is the recommended model for Spaces deployment (no Unsloth/PEFT dependency)
• —Offline clinics: Deploy via Ollama or llama.cpp on consumer hardware (text reasoning; vision requires transformers)
• —Telemedicine platforms: Integrate via Gradio API or transformers pipeline
• —Research: Baseline for chest X-ray screening in low-resource settings

Out-of-Scope Use

• —Not a diagnostic tool. This is an AI screening assistant. All findings must be confirmed by a qualified medical professional.
• —Not validated for: CT scans, MRI, ultrasound, or non-chest radiographs.
• —Not intended for: Autonomous clinical decision-making without human oversight.

Bias, Risks, and Limitations

• —Dataset bias: Trained on NIH ChestX-ray14, which over-represents US hospital populations. Performance may vary on radiographs from different demographics, equipment, or imaging protocols.
• —Label noise: NIH labels are NLP-extracted from radiology reports (~15-20% estimated error rate), not radiologist-annotated. This limits ceiling performance, especially for Pneumonia and Consolidation.
• —False positives: The model tends to over-detect Pneumonia (382 FP / 1103 test) and Consolidation (375 FP / 1103 test). In clinical use, this means unnecessary referrals rather than missed diagnoses.
• —Pneumonia detection is weak: AUC 0.617 on NIH, 0.501 on CheXpert (only 11 positives = insufficient statistical power). Active development.
• —Single-view only: Trained on frontal (PA/AP) chest X-rays. Lateral views not supported.
• —Vision via GGUF not supported: The GGUF export does not include the vision encoder (mmproj). Image analysis requires the transformers library.

Recommendations

• —Always use with clinical oversight — this is a screening aid, not a replacement for radiologists.
• —Review false positives carefully before clinical action.
• —For Pneumonia specifically, treat model output as low-confidence and prioritize clinical judgment.
• —Validate on your target population before deployment.

Training Details

Training Data

• —Source: NIH ChestX-ray14 (112,120 frontal chest X-rays, 30,805 patients, CC0/Public Domain)
• —Pathologies trained: Pneumonia, Consolidation, Cardiomegaly, Pleural Effusion, Pulmonary Edema
• —Training split: ~23,000 examples (from 8,821 base images with oversampling + augmentation)
• —5x oversampling for Pneumonia and Consolidation (rare positives)
• —3x oversampling for Cardiomegaly
• —Augmentation: brightness, contrast, rotation
• —Label format: Conversation-style (image + structured YES/NO per pathology with radiological descriptions)
• —Response length: Short (~80-120 tokens per response)

Training Procedure

Preprocessing

• —Images resized and normalized per Gemma 4 processor defaults
• —Conversation format with 5 varied prompt templates per pathology
• —Dataset v5: oversampled + augmented, balanced for rare positives

Training Hyperparameters

Speeds, Sizes, Times

• —Training time: 4 hours 27 minutes (~16,000 seconds)
• —Steps: ~5,800
• —Speed: ~2.9 samples/sec
• —Hardware: NVIDIA RTX 5070 Ti (16GB VRAM)
• —Peak VRAM: ~10.7 GB
• —Final loss: ~0.089 (avg 0.2009)
• —Trainable parameters: ~82M / 8B total (1.02%)
• —Merged model size: ~16 GB (fp16)

Evaluation

Testing Data, Factors & Metrics

Testing Data

1. 1.NIH ChestX-ray14 held-out test set: 1,103 images with NLP-extracted ground truth labels
2. 2.CheXpert gold standard: 500 images annotated by 5 board-certified radiologists (Stanford)

Metrics

• —AUC (Area Under ROC Curve): Primary metric, threshold-independent discrimination ability
• —Sensitivity (Recall): Proportion of true positives correctly identified
• —Specificity: Proportion of true negatives correctly identified
• —Accuracy: Overall correct classification rate

Results

NIH Test Set (N=1,103 held-out images)

3/5 pathologies exceed AUC 0.70. All 5 improved vs. baseline Gemma 4.

CheXpert Gold Standard (N=500, 5-radiologist consensus, Stanford)

\Pneumonia: only 11 positives (2.2% prevalence) in CheXpert test set — insufficient statistical power.*

Highlight: Pleural Effusion sensitivity of 95.2% on CheXpert — catches 95 out of 100 cases.

Summary

This model demonstrates that fine-tuning Gemma 4 E4B on real clinical images produces genuine visual understanding (not text memorization). The base model scored near-random (AUC ~0.50) on Cardiomegaly; after fine-tuning, it achieves 0.832 — a 70% improvement validated on independent test sets.

Environmental Impact

• —Hardware: 1x NVIDIA RTX 5070 Ti (16GB, consumer GPU)
• —Total GPU hours: ~43 hours (training 18.7h + evaluation 22.4h + misc 2h)
• —Training-only hours: 4.4 hours (v4 final run)
• —Cloud Provider: None (local workstation)
• —Total project cost: < $25 (electricity only)
• —Carbon Emitted: Estimated ~4.3 kg CO2eq (based on Colombia grid factor ~0.1 kg CO2/kWh, RTX 5070 Ti TDP 300W)

Technical Specifications

Model Architecture and Objective

• —Base model: Google Gemma 4 E4B-it (8B parameters with 4.5B effective, vision-language)
• —Fine-tuning method: QLoRA via Unsloth (4-bit quantized base + low-rank adapters), then merged to fp16
• —LoRA rank: 64 on all linear layers (vision + language + attention + MLP)
• —Context length: 128K tokens (inherited from Gemma 4)
• —Objective: Supervised fine-tuning (SFT) on chest X-ray analysis conversations

Compute Infrastructure

Hardware

• —NVIDIA RTX 5070 Ti 16GB (local workstation)
• —64GB system RAM
• —Arch Linux

Software

• —Python 3.14
• —PyTorch 2.10.0+cu128
• —Transformers >= 4.45.0
• —Unsloth (for training + merge)
• —TRL (SFTTrainer)

Training Iterations

This model is the result of 6 training iterations:

Each failure taught us something: long responses dilute gradient signal, low LoRA rank lacks capacity, and clean labels from a different distribution can hurt rather than help.

Citation

BibTeX:

@misc{arias2026medvisionedge,
  title={MedVision Edge: AI Radiology for Everyone},
  author={Arias, Joel},
  year={2026},
  howpublished={\url{https://huggingface.co/j4rias/medvision-edge-v4-merged}},
  note={Fine-tuned Gemma 4 E4B for chest X-ray screening (merged fp16). Gemma 4 Good Hackathon submission.}
}

Acknowledgements

• —Google for the Gemma 4 model family and the Gemma 4 Good Hackathon
• —Unsloth for efficient QLoRA fine-tuning of vision-language models
• —NIH Clinical Center for the ChestX-ray14 dataset (CC0)
• —Stanford AIMI for the CheXpert gold-standard test set
• —WHO for the IMCI clinical protocols

Framework Versions

• —Transformers >= 4.45.0
• —PyTorch 2.10.0+cu128
• —Unsloth (used for training + merge)
• —TRL (latest)
• —PEFT 0.19.1 (used for training)