Datasetpublic

Jinxing1/MQ-RAVSBench

MQ-RAVSBench MQ-RAVSBench is a benchmark for mask-quality auditing in referring audio-visual segmentation. Each example links a video clip, audio, a referring expression, the ground-truth object mask, and candidate masks with different error patterns. The benchmark is used by MQ-Auditor to assess whether a candidate mask should be accepted, revised, or rejected. All paths stored in the metadata files are relative to the dataset root. Dataset Layout MQ-RAVSBench/… See the full description on the dataset page: https://huggingface.co/datasets/Jinxing1/MQ-RAVSBench.

sourceHugging Facecc-by-nc-sa-4.0updated 4mo agoView on Hugging Face

2likes12kdownloads

Dataset Card

MQ-RAVSBench

MQ-RAVSBench is a benchmark for mask-quality auditing in referring audio-visual segmentation. Each example links a video clip, audio, a referring expression, the ground-truth object mask, and candidate masks with different error patterns. The benchmark is used by MQ-Auditor to assess whether a candidate mask should be accepted, revised, or rejected.

All paths stored in the metadata files are relative to the dataset root.

Dataset Layout

text

MQ-RAVSBench/
  README.md
  media/
    <vid>/
      audio.wav
      frames/
        0.jpg ... 9.jpg
  gt_mask/
    <vid>/fid_<fid>/<frame>.png
  part_neg_masks/
    <save_id>/<frame>/{cutout,erode,dilate,merge}/...
  full_neg_masks/
    <save_id>/<frame>/<category>/...
  null_masks/
    <save_id>/<frame>/000.png
  train_test_meta_files/
    metadata.csv
    train_audit_only_filtered.json
    test_s_image_filtered.json
    test_u_image_filtered.json
    test_s_video_filtered.json
    test_u_video_filtered.json

Directory summary for this release:

Directory	Contents
`media/`	1,840 clips, each with `audio.wav` and 10 extracted frames
`gt_mask/`	Ground-truth segmentation masks (`perfect`)
`part_neg_masks/`	Partially incorrect masks, including `cutout`, `erode`, `dilate`, and `merge` errors
`full_neg_masks/`	Masks of non-target objects (`full_neg`)
`null_masks/`	Empty masks used during MQ-Auditor training (`null`)
`train_test_meta_files/`	CSV and JSON metadata used by training and evaluation scripts

Metadata

train_test_meta_files/metadata.csv contains the base sample metadata:

Column	Meaning
`vid`	Clip id. Matches a folder under `media/`
`uid`	Query/object instance id
`split`	Split label from the source metadata
`fid`	Object/mask id used in `gt_mask/<vid>/fid_<fid>/...`
`exp`	Referring expression
`kfid`	Key-frame index used by image-mode evaluation

Split counts in metadata.csv:

Split	Count
`train`	1,306
`test_s`	437
`test_u`	303

The default MQ-Auditor training and evaluation scripts use these JSON files:

File	Entries	Usage
`train_audit_only_filtered.json`	1,306	Supervised fine-tuning data with audit responses
`test_s_image_filtered.json`	437	Seen-category image/key-frame evaluation
`test_u_image_filtered.json`	303	Unseen-category image/key-frame evaluation
`test_s_video_filtered.json`	60	Seen-category video evaluation
`test_u_video_filtered.json`	40	Unseen-category video evaluation

Candidate mask types include perfect, cutout, erode, dilate, merge, full_neg, and null. Candidate entries provide the mask path, IoU to the ground-truth mask, and the recommended audit action when available.

null masks are used when training MQ-Auditor, but they are not part of the default/reported test protocol. In our experiments, the trained auditor can identify this mask type perfectly, so test-time evaluation focuses on the non-empty candidate masks.

Code and Pretrained Weights

The MQ-Auditor source code, training scripts, inference scripts, and pretrained weights are released separately from MQ-RAVSBench: https://github.com/jasongief/MQA-RefAVS

The released MQ-Auditor pretrained checkpoint corresponds to:

text

epochs96_lr1e-4_bs4_gradacc8_lora_r32alpha64_pos0.5_ioulosswei0

See the MQ-Auditor code release for training and evaluation commands.

License

MQ-RAVSBench is licensed under a CC BY-NC-SA 4.0 License and is released for non-commercial research purposes only. MQ-RAVSBench incorporates videos and/or annotations from previous datasets, including Ref-AVS and AVSBench; users must also comply with the licenses and terms of the original datasets.

Citation

If you use MQ-RAVSBench or MQ-Auditor, please cite:

bibtex

@article{zhou2026audit,
  title={Audit After Segmentation: Reference-Free Mask Quality Assessment for Language-Referred Audio-Visual Segmentation},
  author={Zhou, Jinxing and Zhou, Yanghao and Wang, Yaoting and Han, Zongyan and Ma, Jiaqi and Ding, Henghui and Anwer, Rao Muhammad and Cholakkal, Hisham},
  journal={arXiv preprint arXiv:2602.03892},
  year={2026}
}

Paper: https://arxiv.org/pdf/2602.03892