radar-generalist/RADAR-auxiliary-data

RADAR: Preprocessed Anatomical Masks for Merlin CT Data

   

This dataset provides preprocessed anatomical segmentation masks for the Merlin abdominal CT training set, generated by TotalSegmentator and post-processed for use with the RADAR framework. These masks enable anatomy-aware vision–language pretraining without any additional manual annotation.

Overview

RADAR is a generalist vision–language model trained on over 400,000 contrast-enhanced abdominal CT examinations with 15 million anatomy-aware image–text pairs. A key component of its training pipeline is the use of anatomical segmentation masks to establish spatial correspondences between image regions and organ-level textual descriptions. This dataset releases the preprocessed masks used in the RADAR+ experiments on the Merlin dataset.

Mask Generation Pipeline

The masks were produced through the following steps:

1. 1.Automatic segmentation: Each Merlin training CT volume was processed with TotalSegmentator, producing voxel-level labels for 104 anatomical structures.
2. 2.Structure merging: The 104 fine-grained labels were consolidated into 36 main anatomical structures relevant to abdominal CT diagnosis.
3. 3.Spacing resampling: Masks were resampled to a uniform spacing of 1 × 1 × 5 mm (matching the resampled CT images).
4. 4.Format: Each mask is saved as a single-channel NIfTI file (.nii.gz), where each voxel value corresponds to an organ index (1–36; 0 = background).

36 Anatomical Structures

Organ Index Table

Dataset Structure

data/merlin_data_train_full/resized_masks/
├── part_00/
│   ├── <patient_id_a>.nii.gz
│   └── ...
├── part_01/
│   ├── <patient_id_b>.nii.gz
│   └── ...
└── part_02/
    ├── <patient_id_c>.nii.gz
    └── ...

• —Each .nii.gz file shares the same patient ID as the corresponding Merlin CT image.
• —The mask files are aligned 1-to-1 with the resampled images in resized_images/ (not included here; see Merlin dataset for the original CT volumes).
• —Before using the dataset, you need to consolidate the files into the resized_masks/ directory. Run the following command in your terminal from the root of the project:

# Navigate to the target directory
cd data/merlin_data_train_full/resized_masks/

# Move all .nii.gz files from subdirectories to the current folder
mv part_*/ *.nii.gz .

# Optional: Remove the empty part directories
rm -rf part_*/

Usage with RADAR

Directory Layout

Place the downloaded masks alongside the Merlin images to form the expected directory structure:

radar/data/merlin_data_train_demo/       # or merlin_data_train_full/
├── resized_images/
│   ├── <patient_id>.nii.gz              # Merlin CT volumes resampled to 1×1×5 mm
│   └── ...
└── resized_masks/
    ├── <patient_id>.nii.gz              # ← This dataset
    └── ...

How Masks Are Used in Training

During training, the dataloader (caption_datasets.py):

1. 1.Loads each CT image and its corresponding mask via MONAI transforms.
2. 2.Pads and center-crops both to a fixed size of 96 × 256 × 384 voxels.
3. 3.Identifies intact organs (those whose mask regions are fully contained within the crop, not truncated at boundaries).
4. 4.Pairs each intact organ region with its organ-level clinical report text, enabling anatomy-aware contrastive learning.

Data Files

Prerequisites

The anatomical masks in this dataset are designed to be used together with the original Merlin CT images. You will need to:

1. 1.Download the Merlin dataset and resample the CT volumes to 1 × 1 × 5 mm spacing.
2. 2.Download the RADAR model checkpoints from HuggingFace.

Citation

If you use these masks in your research, please cite:

@article{damo-radar-2026,
    author = {Qi Zhang and Jianpeng Zhang and Weiwei Cao and Zilin Lu and Wanxing Chang and Haonan Ding and Cao Chen and Zhi Li and Xing Xue and Sinuo Wang and Shaoteng Zhang and Yutong Xie and Yong Xia and Qi Wu and Zhongyi Shui and Xi Li and Zhilin Zheng and Yanjie Zhou and Tony C.W. Mok and Yingda Xia and Hongkan Wang and Xianghua Ye and Tao Ma and Jie Peng and Xiaoguang Wang and Jian Ding and Yuming Gao and Huazhen Ye and Yiping Liu and Dongjie Chen and Zhaomin Ni and Jianwen Ning and Wei Zhang and Jian Liu and Chaohui Yu and Shenghong Ju and Jianfeng Zhang and Wenbo Xiao and Ling Zhang and Tingbo Liang },
    title = {An expert-level generalist AI for abdominal CT diagnosis},
    journal = {Science},
    volume = {393},
    number = {6817},
    pages = {eaec6129},
    year = {2026},
    doi = {10.1126/science.aec6129},
    URL = {https://www.science.org/doi/abs/10.1126/science.aec6129}
}

License

This dataset is released under CC BY-NC-SA 4.0.