Modelpublic

ahmet-harun/turkradllm-cervical

sourceHugging Faceotherupdated 16d agoView on Hugging Face

0likes30downloads

Model Card

TurkRadLLM-Cervical

LoRA adapter for google/medgemma-4b-it that converts free-text Turkish cervical spine MRI reports into a structured six-field JSON representation.

This adapter accompanies the study TurkRadLLM: A Compact MedGemma-Based Model for Structuring Turkish Lumbar and Cervical Spine MRI Reports — A Multi-Reader Validation Study (under review). It is released for research reproducibility.

Research use only. This model has not been validated or cleared for clinical use as a medical device. See Intended use and restrictions below.

Output schema

json

{
  "modality": "MR",
  "body_part": "Servikal Omurga",
  "positive_findings": ["..."],
  "negative_findings": ["..."],
  "impression": "...",
  "urgent_flag": false
}

Field definitions

Field	Meaning
`modality`	Imaging modality, as stated or implied by the report header (here always `MR`).
`body_part`	Anatomical region covered by the examination.
`positive_findings`	Pathological findings, one list item per finding, with level and laterality where the report states them.
`negative_findings`	Structures explicitly described as normal, or pathologies explicitly excluded. Extraction here is deliberately conservative.
`impression`	A single summary sentence corresponding to the overall assessment.
`urgent_flag`	Whether the report contains a finding requiring urgent action (see `PROMPT.md` for the criteria).

Model details

Property	Value
Base model	`google/medgemma-4b-it` (Gemma 3 architecture)
Method	QLoRA (4-bit NF4 base, bfloat16 compute)
LoRA rank / α	8 / 16
Target modules	`q_proj`, `k_proj`, `v_proj`, `o_proj` (attention only)
Trainable parameters	5,949,440 (adapter ≈24 MB)
Max sequence length	2,048
Training data	5,017 de-identified Turkish cervical spine MRI reports (single tertiary centre, 2020–2025)
Split	4,013 train / 502 validation / 502 held-out test, stratified by urgent-finding prevalence
Final validation loss	0.145
Training hardware / time	Single NVIDIA A100-SXM4-80GB, 82.4 min

Adaptation was restricted to the attention projections because broader adaptation (attention together with the MLP projections) reproducibly produced degenerate multilingual output during development.

Evaluation

Reported on the complete held-out test set (n = 502 reports) unless stated otherwise. Automated metrics quantify agreement with silver-standard labels and are a measure of label fidelity, not of clinical accuracy; clinical acceptability was assessed separately by radiologists.

Metric	This adapter	Base MedGemma 4B (zero-shot)
Valid, schema-conformant JSON	100%	100%
modality	99.2%	99.2%
body_part	100%	100%
positive_findings F1	0.693	0.579
negative_findings F1	0.692	0.531
impression correct	59.8%	17.9%
urgent flag, sensitivity	67.6% (25/37)	21.6% (8/37)
urgent flag, specificity	93.5%	93.1%

How to read these numbers

—Valid JSON — the output parses and conforms to the six-field schema.
—modality / body_part — exact-match agreement. These fields are largely invariant across the corpus, so a trivial rule-based baseline also scores near-ceiling on them; they do not discriminate between systems.
—positive_findings / negative_findings F1 — list-level F1 computed with token-overlap matching between predicted and reference items, averaged per report.
—impression correct — proportion of reports whose summary sentence matches the reference assessment under lenient token-overlap matching.
—urgent flag — sensitivity is computed on the urgent reports in the held-out set, specificity on the remainder. An always-negative classifier would reach high accuracy at zero sensitivity, so accuracy alone is not informative for this field.

A multi-reader validation of 180 held-out reports (both anatomical regions pooled), scored field by field by two radiology residents with discordances adjudicated by a consultant radiologist, is reported in the accompanying paper.

Known weaknesses. The urgent flag has limited sensitivity and is unsuitable for autonomous triage. In the multi-reader study, negative-finding completeness was reader-dependent. The schema does not provide dedicated structured fields for measurements, laterality, or vertebral-level localisation. Training data come from a single centre and a single reporting culture; no external or prospective validation has been performed.

Deployment footprint

Measured with vLLM (bfloat16, greedy decoding) on a consumer NVIDIA RTX 4090 (24 GB) using the lumbar adapter, which is identical in size and architecture to this one: peak GPU memory 14.75 GB, median single-report latency 1.66 s, batch throughput 1,345 reports/min. The adapter itself is ≈24 MB.

Usage

python

from transformers import AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel

base = AutoModelForImageTextToText.from_pretrained(
    "google/medgemma-4b-it", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "ahmet-harun/turkradllm-cervical")
processor = AutoProcessor.from_pretrained("google/medgemma-4b-it")

The evaluation prompt used in the study is provided in PROMPT.md in this repository, for reproducibility. Performance with alternative prompts was not evaluated.

Intended use and restrictions

Intended use. Research on clinical natural-language processing, reproduction of the accompanying study, and benchmarking. Intended for researchers and developers, not for patients or for clinicians making care decisions.

Out-of-scope use. The following are outside the intended use and are not supported:

—primary or assistive clinical decision-making, triage, or patient-facing deployment;
—deployment on patient data without appropriate institutional governance, privacy and security review, local validation, and any regulatory authorisation required in the relevant jurisdiction;
—use as a medical device, or incorporation into one, without the regulatory clearance required in the relevant jurisdiction.

Performance reported here was measured at a single centre on a single reporting culture and should not be assumed to transfer; users deploying the adapter elsewhere should validate it locally.

Licence. This model is a derivative of google/medgemma-4b-it and is distributed subject to the Health AI Developer Foundations Terms of Use, which continue to apply in full to this adapter. See the NOTICE file in this repository. Users are bound by the base-model terms as well as by the restrictions above.

Attribution. Use in published work should cite both this adapter and the base MedGemma model.

Data. The underlying radiology reports are protected health information and are not released. The prompt, the JSON schema and the scoring rubric are released. The silver labels were generated by a proprietary large language model and were used as training targets and as the reference for the automated fidelity analyses; they were not used as the clinical reference standard in the multi-reader validation.

Ethics

The study was approved by the Scientific Research Ethics Committee of Ümraniye Training and Research Hospital, University of Health Sciences, Istanbul, Türkiye (approval no. 272; 18 June 2026). The requirement for written informed consent was waived owing to the retrospective use of de-identified reports; handling of anonymised data complied with the Turkish Personal Data Protection Law (KVKK).

Citation

bibtex

@article{turgan2026turkradllm,
  title   = {TurkRadLLM: A Compact MedGemma-Based Model for Structuring Turkish Lumbar and
             Cervical Spine MRI Reports — A Multi-Reader Validation Study},
  author  = {Turgan, Ahmet Harun and Çolakoğlu, Hasan and Demir, Akif and Sakcı, Zakir},
  journal = {Journal of Imaging Informatics in Medicine},
  year    = {2026},
  note    = {Under review}
}

Base model: