lphuang33/EG-VQA

EG-VQA

   

This repository provides the videos and annotations of EG-VQA. EG-VQA is an evidence-grounded, open-ended Video Question Answering benchmark. It contains 2,067 videos and 11,838 question-answer pairs. Each question is paired with one or more temporally localized evidence segments, including timestamps and textual descriptions.

Questions cover four types:

• —Descriptive: recognizing and describing visible events or states.
• —Temporal: reasoning about event order, timing, and temporal relations.
• —Causal: identifying causes, effects, and event dependencies.
• —Counterfactual: reasoning about hypothetical changes to observed events.

Data Sources

EG-VQA is constructed from videos originating from three existing video-language datasets.

The train/test split is performed at the video level, so the same video never appears in both splits.

Dataset Structure

EG-VQA/
├── videos/
│   ├── <video_id>.mp4
│   ├── <video_id>.mkv
│   └── ...
├── train.json
├── test.json
└── README.md

Both train.json and test.json are JSON lists. Each top-level record corresponds to one video and contains the video metadata, source temporal segments, and all EG-VQA questions associated with that video.

Annotation Format

A typical record has the following structure:

{
  "video_id": "example_video_id",
  "video_path": "videos/example_video_id.mp4",
  "duration": 120.0,
  "data_source": "YouCook2",
  "metadata": {
    "title": "Example instructional video",
    "segments": [
      {
        "timestamp": [12.0, 24.5],
        "description": "The person prepares an object on the work surface."
      }
    ]
  },
  "questions": [
    {
      "question_id": "example_video_id_q01",
      "question": "What does the person do after preparing the object?",
      "type": "temporal",
      "answer": "The person places it on the work surface.",
      "evidence": [
        {
          "timestamp": [24.5, 31.0],
          "description": "The prepared object is placed on the work surface."
        }
      ]
    }
  ]
}

Field Description

Video-level fields

Question-level fields

License

The EG-VQA annotations are released under CC BY 4.0. The underlying source videos and source-dataset content remain subject to their respective original licenses and terms of use.

Citation

If you find EG-VQA useful for your research, please cite our paper:

@article{huang2026egvqa,
  title={EG-VQA: Benchmarking Verifiable Video Question Answering with Grounded Temporal Evidence},
  author={Huang, Linpeng and Chen, Weixing and Chen, Zexin and Liu, Yang and Lin, Liang},
  journal={arXiv preprint arXiv:2606.24797},
  year={2026}
}

Acknowledgements

EG-VQA builds on videos and annotations from ActivityNet Captions, YouCook2, and HiREST. We thank the authors and contributors of these datasets for making their resources available to the research community.