inesriahi/valor32k-avqa-v2

Valor32k-AVQA v2.0

Valor32k-AVQA v2.0 is an open-ended audio-visual question answering dataset and benchmark with 28,861 videos and 225,487 question-answer pairs in this Hugging Face release. Each question is annotated with a modality label (visual, audio, or audio-visual) and one of six categories: description, action, count, temporal, location, and relative-position.

Links

• —Paper: ACM Digital Library
• —Project page: inesriahi.github.io/valor32k-avqa-2
• —Code and documentation: GitHub repository

Modalities and Format

• —Modalities: text annotations and tabular metadata for audio-video QA samples. Test split videos are included under videos/test/; train and validation media are referenced by video_id but are not stored as media files in this Hugging Face release.
• —Tasks: audio-visual question answering, video question answering, visual question answering
• —Formats: JSONL on Hugging Face; original JSON release files are available from the project repository
• —Library: compatible with datasets.load_dataset

The Hugging Face release provides the flattened QA format as the default configuration:

• —default: one question-answer item per row. This is the recommended format for training, evaluation, and Dataset Viewer preview.

The modality column can be used to filter examples into visual, audio, and audio-visual subsets without duplicating the dataset into separate configurations.

Test Videos

The test split videos are included under videos/test/ as raw .mp4 files. The companion videos/test/metadata.jsonl file follows Hugging Face VideoFolder conventions: each row contains the flattened QA fields plus a file_name field that points to the corresponding MP4 in the same directory.

The default dataset configuration remains the flattened JSONL QA table under data/flattened/. To load the test videos with their QA metadata:

from datasets import load_dataset

test_videos = load_dataset(
    "inesriahi/valor32k-avqa-v2",
    data_dir="videos",
    split="test",
    streaming=True,
)

# Optional: get remote video paths without requiring torchcodec decoding.
test_videos = test_videos.decode(False)

To use the flattened QA table and resolve videos manually, join rows by video_id to videos/test/metadata.jsonl or use the file_name field in that metadata file.

Loading

from datasets import load_dataset

dataset = load_dataset("inesriahi/valor32k-avqa-v2")
train = dataset["train"]

Flattened Schema

• —caption: source video caption
• —video_id: video identifier
• —question: generated question
• —options: four answer choices
• —correct_answer_idx: zero-based index of the correct option
• —rephrased_answers: three paraphrases of the correct answer
• —quality_rating: obvious or guess
• —modality: visual, audio, or audio-visual
• —category: question category
• —source_tags: generation cues such as frames, audio, and caption
• —id: question id
• —oid: original video-level group id
• —model: generator model label

Dataset Statistics

• —Unique videos: 28,861
• —Total questions: 225,487
• —Train questions: 177,132
• —Validation questions: 22,267
• —Test questions: 26,088

Questions by Modality

• —Visual: 130,640
• —Audio-Visual: 49,663
• —Audio: 45,184

Questions by Category

• —Description: 62,701
• —Action: 47,689
• —Count: 41,896
• —Temporal: 41,358
• —Location: 19,245
• —Relative Position: 12,598

Notes

This Hugging Face release includes the flattened QA annotations and the test split videos under videos/test/. Train and validation source videos are not redistributed in this Hugging Face dataset. Additional sample clips and project documentation are available in the project repository.

Citation

@inproceedings{riahi2025valor32k,
  author    = {Riahi, Ines and Radman, Abduljalil and Guo, Zixin and Hedjam, Rachid and Laaksonen, Jorma},
  title     = {Valor32k-AVQA v2.0: Open-Ended Audio-Visual Question Answering Dataset and Benchmark},
  booktitle = {Proceedings of the 33rd ACM International Conference on Multimedia},
  series    = {MM '25},
  year      = {2025},
  pages     = {13097--13103},
  publisher = {Association for Computing Machinery},
  address   = {New York, NY, USA},
  location  = {Dublin, Ireland},
  isbn      = {9798400720352},
  doi       = {10.1145/3746027.3758261},
  url       = {https://doi.org/10.1145/3746027.3758261}
}

License

Research use only. See the project repository for the full license terms.

Copyright / Takedown

We respect the rights of content creators and copyright holders. If you believe that any video included in this dataset infringes your rights or should not be included, please contact us. We will remove the corresponding content from future releases of the dataset.