aehrc/cxrmate-2

CXRMate-2 — Chest X-ray Radiology Report Generation

From: CXRMate-2: Structured Multimodal Temporal Embeddings and Tractable Reinforcement Learning for Clinically Acceptable Chest X-ray Radiology Report Generation

<!-- CXRMate-2 is presented in the following Medical Image Analysis paper: --> <!-- ```

The pre-print for CXRMate-2 is available at:

@misc{nicolson2026clinicallyacceptablechestxray, title={CXRMate-2: Structured Multimodal Temporal Embeddings and Tractable Reinforcement Learning for Clinically Acceptable Chest X-ray Radiology Report Generation}, author={Aaron Nicolson and Elizabeth J. Cooper and Hwan-Jin Yoon and Claire McCafferty and Ramya Krishnan and Michelle Craigie and Nivene Saad and Jason Dowling and Ian A. Scott and Bevan Koopman}, year={2026}, eprint={2604.18967}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2604.18967}, }

## Load CXRMate-2:

alias = 'aehrc/cxrmate-2'

model = transformers.AutoModelForCausalLM.frompretrained(alias, trustremote_code=True).to(device='cuda') model.eval()

generationconfig = transformers.GenerationConfig.frompretrained(alias, trustremotecode=True)

processor = transformers.AutoProcessor.frompretrained(alias, trustremote_code=True)

## Generate from URL:

url = 'https://prod-images-static.radiopaedia.org/images/220869/76052f7902246ff862f52f5d3cd9cdbiggallery.jpg'

display(Image(url=url))

processed = processor(images=url) processed = processed.to(device='cuda')

generatedids = model.generate(**processed, generationconfig=generation_config)

findings, impression = processor.splitanddecodesections(generatedids) print(f'Findings:\t{findings[0]}\nImpression:\t{impression[0]}')

## Generate from study DICOMs:

dcmpath = [ './physionet.org/files/mimic-cxr/2.0.0/files/p12/p12000264/s55271473/522f9570-7cb12ecb-6327c8b8-b248b4be-58bb3dfd.dcm', './/datasets/work/hb-mlaifsp-mm/work/repositories/25cxrmate2/work/data/physionet.org/files/mimic-cxr/2.0.0/files/p12/p12000264/s55271473/d0b61aff-f64c4ecf-85fae310-43668cf1-0c1d4c2d.dcm', ]

processed = processor(images=dcm_path) processed = processed.to(device='cuda')

generatedids = model.generate(**processed, generationconfig=generation_config)

findings, impression = processor.splitanddecodesections(generatedids) print(f'Findings:\t{findings[0]}\nImpression:\t{impression[0]}')

## Generate from ReXgradient HuggingFace dataset:
**Scripts to create the ReXgradient, CheXpert Plus, and MIMC-CXR Hugging Face datasets will be released soon**

Run preparechexpertplus.py or preparemimiccxrjpg.py or preparerexgradient.py to create dataset:

testset = datasets.loadfromdisk('/scratch3/nic261/database/cxrmate2/rexgradient160k_dataset')['test']

Wrap dataset to get priors:

testset = processor.wrapdataset(test_set)

randomidx = random.randint(0, len(testset) - 1) example = testset[randomidx]

processed = processor( images=example['images'], # This includes both the current and prior images. imagedatetime=example['imagedatetime'], # This includes the datetimes for both the current and prior images. views=example['views'], # This includes both the current and prior views. indication=example.get('indication', None), history=example.get('history', None), comparison=example.get('comparison', None), technique=example.get('technique', None), studydatetime=example.get('studydatetime', None), priorfindings=example.get('priorfindings', None), priorimpression=example.get('priorimpression', None), priorstudydatetime=example.get('priorstudydatetime', None), ) processed = processed.to(device='cuda')

generatedids = model.generate(**processed, generationconfig=generation_config)

findings, impression = processor.splitanddecodesections(generatedids) print(f'Findings:\t{findings[0]}\nImpression:\t{impression[0]}')

# Environment requirements:

Environment requirements can be found here: https://github.com/aehrc/cxrmate-ed/blob/main/requirements.txt.


# Training:

The training pipeline for CXRMate-2, is available at: https://github.com/aehrc/cxrmate-2.