Panorama-grounding/MRSeg-Referring-Expressions
MRSeg Referring Expressions (single-turn) Single-turn referring-expression segmentation samples derived from the multi-round MR-Seg data of SegLLM (paper). SegLLM's original conversations provide the target regions as input to later rounds, which makes them unusable for models that receive only an image and text. We used Qwen3-VL-32B-Instruct to keep only the samples whose target can be identified from the image and expression alone, rewriting each into a self-contained… See the full description on the dataset page: https://huggingface.co/datasets/Panorama-grounding/MRSeg-Referring-Expressions.
MRSeg Referring Expressions (single-turn)
   
Single-turn referring-expression segmentation samples derived from the multi-round MR-Seg data of SegLLM (paper).
SegLLM's original conversations provide the target regions as input to later rounds, which makes them unusable for models that receive only an image and text. We used Qwen3-VL-32B-Instruct to keep only the samples whose target can be identified from the image and expression alone, rewriting each into a self-contained referring expression. The segmentation masks are resolved from the source datasets' annotations and stored inline as COCO RLE, so no external annotation files are needed — only the images.
The release contains 101,349 samples spanning parts, stuff regions, attributes, and relationally specified regions. This is the multi-granularity bucket of PANORAMA's training mixture.
See also our human-annotated benchmark for panoptic grounded captioning, PanoCaps, and our re-captioned COCONut-PanCap.
What is in this repository
Download
hf download Panorama-grounding/MRSeg-Referring-Expressions --repo-type dataset \
--local-dir /path/to/data/MRSeg-Referring-Expressions
mkdir -p /path/to/data/mrseg_extract && tar -xf /path/to/data/MRSeg-Referring-Expressions/mrseg_accepted.tar \
-C /path/to/data/mrseg_extract # accepted/*.jsonlImages
Images are not included. Samples reference them by a relative path (img_rel) under four public sources:
LVIS, PACO and COCO-Stuff annotate COCO images, so they require no additional image download.
Fields
License
This dataset is derived from SegLLM's MR-Seg data and from the annotations of LVIS, PACO, Visual Genome, COCO-Stuff, ADE20K and PASCAL-Part; it follows the licenses of these source datasets. For research and non-commercial use only.
Citation
If you use these expressions, please cite our paper and the original SegLLM work:
@article{pieri2026panorama,
title = {PANORAMA: Panoptic Grounded Captioning via Mask Proposal Selection},
author = {Pieri, Sara and Kazakos, Evangelos and Chen, Shizhe and Sivic, Josef and Schmid, Cordelia},
journal = {arXiv preprint arXiv:2609.19143},
year = {2026}
}
@inproceedings{wang2025segllm,
title = {{SegLLM}: Multi-round Reasoning Segmentation with Large Language Models},
author = {Wang, XuDong and Zhang, Shaolun and Li, Shufan and Kallidromitis, Konstantinos and Li, Kehan and Kato, Yusuke and Kozuka, Kazuki and Darrell, Trevor},
booktitle = {International Conference on Learning Representations},
year = {2025}
}