shuzhig/nextgqa

NExT-GQA (mirror)

A redistribution of the NExT-GQA benchmark, packaged as a single self-contained repo (annotations + the 1,570 videos needed to run it) for convenience.

This is not the official release. All credit goes to the original authors. Official code and data: https://github.com/doc-doc/NExT-GQA

Dataset description

NExT-GQA extends NExT-QA with temporal grounding labels: for each multiple-choice question it annotates the video segment(s) that actually justify the answer. It is designed to test whether a video-QA model answers because it located the right evidence, or by exploiting language priors.

Only the val and test splits carry grounding labels, so those are what this repo contains.

Contents

annotations/
  test.csv               # 5,553 QA rows  (video_id, question, answer, qid, type, a0..a4)
  val.csv                # 3,358 QA rows  (same schema)
  gsub_test.json         # temporal grounding labels for test
  gsub_val.json          # temporal grounding labels for val
  map_vid_vidorID.json   # video_id -> original VidOR ID (e.g. "0101/2909445186")
  nextgqa_test_tg.json   # test split flattened for temporal-grounding evaluation
videos/                  # 1,570 .mp4 files, named <video_id>.mp4

test.csv / val.csv

A question is uniquely identified by the (video_id, qid) pair.

gsub_*.json

Keyed by video_id:

"10001787725": {
  "duration": 34,
  "fps": 29.97,
  "location": {
    "1": [[1.2, 5.8]],
    "3": [[12.1, 17.1], [20.0, 23.5], [29.7, 33.2]]
  }
}

location maps a qid to one or more [start, end] intervals in seconds that ground the answer. A question may have several disjoint supporting segments.

nextgqa_test_tg.json

The test split flattened into one record per question, in the layout expected by common temporal-grounding evaluation code:

{
  "video": "videos/10109006686.mp4",
  "duration": 35.0,
  "timestamp": [0.5, 12.6],
  "sentence": "why are there three people standing beside the red net fence",
  "qid": "nextgqa_10109006686_0",
  "pred": [],
  "video_start": null,
  "video_end": null
}

pred is an empty placeholder for model predictions; video_start / video_end are unset (the full clip is used). Paths in video are relative to the repo root, so they resolve directly against videos/.

Usage

from huggingface_hub import snapshot_download

root = snapshot_download(repo_id="shuzhig/nextgqa", repo_type="dataset")

Annotations only:

snapshot_download(
    repo_id="shuzhig/nextgqa",
    repo_type="dataset",
    allow_patterns="annotations/*",
)

The CSV/JSON files are deliberately left in their upstream shapes rather than reshaped into a datasets config, so existing NExT-GQA evaluation code works against this repo unchanged.

Provenance and licensing

• —Annotations: NExT-GQA, Xiao et al. — see the official repository.
• —Questions and answers derive from NExT-QA; the videos originate from the VidOR dataset.

Licensing follows the upstream releases — consult NExT-GQA, NExT-QA, and VidOR for the terms that apply to your use. This mirror asserts no additional rights, and exists only to make the benchmark reproducible from one download. If you are an author and would like it removed, please open a discussion.

Citation

@inproceedings{xiao2023can,
  title     = {Can I Trust Your Answer? Visually Grounded Video Question Answering},
  author    = {Xiao, Junbin and Yao, Angela and Li, Yicong and Chua, Tat-Seng},
  booktitle = {CVPR},
  year      = {2024}
}

@inproceedings{xiao2021next,
  title     = {NExT-QA: Next Phase of Question-Answering to Explaining Temporal Actions},
  author    = {Xiao, Junbin and Shang, Xindi and Yao, Angela and Chua, Tat-Seng},
  booktitle = {CVPR},
  year      = {2021}
}