Modelpublic

arxp/multimodal-clinical-reasoning

sourceHugging Faceotherupdated 27d agoView on Hugging Face

0likes

Model Card

Multimodal Clinical Reasoning for Chest X-Ray Reports

Research card — no model weights or clinical system are released here. This repository documents an academic deep-learning project and links to its reproducible notebooks, evaluation artifacts and report.

Status at a glance

Item	Availability
Research description	Available
Architecture and ablation results	Available
Training/evaluation notebooks	Companion GitHub repository
Model weights/checkpoints	Not released
MIMIC-CXR data	Not redistributed
Clinical use	Not permitted or validated

Project overview

The project studies image-conditioned radiology report generation with a frozen medical vision encoder and a language decoder. Cross-attention connects chest X-ray patch representations to generated report tokens.

text

Chest X-ray image → BiomedCLIP vision encoder → patch features
                                                    │
                                                    ▼
Report tokens → GPT-2 decoder with cross-attention → generated report

Four controlled variants examine whether attention regularization changes generation quality and alignment behavior:

Variant	Experiment
`baseline`	Cross-attention report generator without alignment regularization
`entropy_only`	Adds attention-entropy regularization
`sparsity_only`	Adds sparsity-oriented attention regularization
`full_alignment`	Combines entropy, sparsity and alignment losses

Evaluation snapshot

The companion repository reports held-out evaluation on 1,000 samples.

Variant	BLEU-1	METEOR	ROUGE-L	CIDEr	Finding overlap
`baseline`	0.1391	0.1359	0.1137	0.0099	0.6429
`entropy_only`	0.1370	0.1344	0.1108	0.0090	0.6732
`sparsity_only`	0.1301	0.1263	0.1094	0.0101	0.5606
`full_alignment`	0.1361	0.1336	0.1105	0.0080	0.6439

The regularized variants did not consistently outperform the baseline across text-generation metrics. The entropy-only variant produced the highest reported finding-overlap score, illustrating why multiple evaluation dimensions are necessary for medical report generation.

Contribution and attribution

This is a collaborative academic project. The source repository's recorded contribution history attributes the following work to Arda Çiftçi (`arxp` / `OxyOxygen`):

—VQA-RAD data cleaning and exploratory/statistical analysis;
—engineering 34 text, image and clinical features across 2,244 cleaned examples;
—dataset documentation and research-summary artifacts.

The public project later evolved into the MIMIC-CXR report-generation and ablation pipeline summarized above. This card distinguishes Arda's recorded contribution from the complete team's final system.

Intended use

This card is intended for:

—inspecting an academic multimodal deep-learning workflow;
—reviewing an ablation-study design;
—understanding the limits of automated radiology report generation;
—navigating to the reproducible notebooks and project report.

It is not an inference endpoint, diagnostic device, clinical decision-support system or substitute for a qualified radiologist.

Limitations and risks

—No clinical validation, prospective study or regulatory review was performed.
—Generated text may omit, hallucinate or misstate clinically important findings.
—BLEU, METEOR, ROUGE and CIDEr do not fully measure clinical correctness.
—Dataset composition and institutional practices may limit generalization.
—MIMIC-CXR access and redistribution remain subject to the dataset provider's terms.
—Checkpoints and source medical data are intentionally excluded from this Hugging Face repository.

Reproducibility resources

Citation

If you reference this educational artifact, cite the companion GitHub repository and its contributors. This Hugging Face page is a research summary, not a released model package.