Jupitern52/TextBraTS
2
1---2 3datasets:4- Jupitern52/TextBraTS5language:6- en7license: mit8pipeline_tag: image-segmentation9library_name: monai10---11 12 13# TextBraTS14 15A volume-level text-image public dataset with novel text-guided 3D brain tumor segmentation from BraTS challenge.16 17---18 19## Introduction20 21**TextBraTS** is an open-access dataset designed to advance research in text-guided 3D brain tumor segmentation. It includes paired multi-modal brain MRI scans and expertly annotated radiology reports, enabling the development and evaluation of multi-modal deep learning models that bridge vision and language in neuro-oncology. Our work has been accepted by MICCAI 2025. The paper is also available on arXiv.22 2324 25## Features26 27- Multi-modal 3D brain MRI scans with expert-annotated segmentation (T1, T1ce, T2, FLAIR) from BraTS20 challenge training set28- Structured radiology reports for each case29- Text-image alignment method for research on multi-modal fusion30 3132 33## Usage34 35You can use this dataset for:36- Developing and benchmarking text-guided segmentation models37- Evaluating multi-modal fusion algorithms in medical imaging38- Research in language-driven medical AI39 40## Installing Dependencies41Run the following commands to set up the environment:42<pre>conda env create -f environment.yml 43pip install git+https://github.com/Project-MONAI/MONAI.git@07de215c </pre>44If you need to activate the environment, use:45<pre>conda activate TextBraTS </pre>46 47## Dataset48 49Due to BraTS official guidelines, MRI images must be downloaded directly from the [BraTS 2020 challenge website](https://www.med.upenn.edu/cbica/brats2020/data.html) (training set).50 51**Download our text, feature, and prompt files:** 52You can download our dataset from [Google Drive](https://drive.google.com/file/d/1i1R6_bVY4VbNtxEIQVsiXUSWuVAtgJhg/view?usp=sharing) or [Hugging Face](https://huggingface.co/datasets/Jupitern52/TextBraTS).53Our provided text reports, feature files, and prompt files are named to match the original BraTS folder IDs exactly. You can set the path and simply merge them with the downloaded MRI data by `merge.py`. 54<pre>python merge.py</pre>55 56If you would like to change the dataset split, please modify the `Train.json` and `Test.json` files accordingly. 57 58## Inference59 60We provide our pre-trained weights for direct inference and evaluation. 61Download the weights from [checkpoint](https://drive.google.com/file/d/147283LL2fRDcTYR_vQA-95vbZysjjD1v/view?usp=sharing).62 63After downloading, place the weights in your desired directory, then run the `test.py` with following command for inference:64 65<pre>python test.py --pretrained_dir=/path/to/your/weights/ --exp_name=TextBraTS</pre>66 67## Training68 69If you would like to train the model from scratch, you can modify the training code `main.py` and please use the following command:70 71<pre>python main.py --distributed --use_ssl_pretrained --save_checkpoint --logdir=TextBraTS</pre>72 73- The `--use_ssl_pretrained` option utilizes the pre-trained weights from NVIDIA's Swin UNETR model.74- Download the Swin UNETR pre-trained weights from [Pre-trained weights](https://drive.google.com/file/d/1FJ0N_Xo3olzAV-oojEkAsbsUgiFsoPdl/view?usp=sharing).75- Please place the downloaded weights in the appropriate directory as specified in your configuration or script.76 77 78## Citation79 80If you use TextBraTS in your research, please cite:81 82```bibtex83@inproceedings{shi2025textbrats,84 title = {TextBraTS: Text-Guided Volumetric Brain Tumor Segmentation with Innovative Dataset Development and Fusion Module Exploration},85 author = {Shi, Xiaoyu and Jain, Rahul Kumar and Li, Yinhao and Hou, Ruibo and Cheng, Jingliang and Bai, Jie and Zhao, Guohua and Lin, Lanfen and Xu, Rui and Chen, Yen-wei},86 booktitle = {Proceedings of the International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI)},87 year = {2025},88 note = {to appear}89}