CoolFace
Datasetpublic

Video-Reason/video-mcp

Video-MCP Video-MCP is a synthetic video dataset for training and evaluating video generation models on multiple-choice question-answering (MCQA) tasks. Each sample is a short video clip (~5 seconds) where a visual question-answering prompt is embedded directly into the video frames, and the correct answer is revealed by progressively highlighting one of four answer boxes (A/B/C/D) over the duration of the clip. The… See the full description on the dataset page: https://huggingface.co/datasets/Video-Reason/video-mcp.

sourceHugging Faceotherupdated 6mo agoView on Hugging Face
2likes440downloads
Dataset Card

Video-MCP

<a href="https://video-reason.com" target="blank"> <img alt="Project Page" src="https://img.shields.io/badge/Project%20-%20Homepage-4285F4" height="20" /> </a> <a href="https://github.com/Video-Reason/VBVR-EvalKit" target="blank"> <img alt="Code" src="https://img.shields.io/badge/Evaluationcode-VBVRBench-100000?style=flat-square&logo=github&logoColor=white" height="20" /> </a> <a href="https://github.com/Video-Reason/VBVR-Wan2.2" target="blank"> <img alt="Code" src="https://img.shields.io/badge/Trainingcode-VBVRWan2.2-100000?style=flat-square&logo=github&logoColor=white" height="20" /> </a> <a href="https://github.com/Video-Reason/VBVR-DataFactory" target="blank"> <img alt="Code" src="https://img.shields.io/badge/Datacode-VBVRDataFactory-100000?style=flat-square&logo=github&logoColor=white" height="20" /> </a> <a href="https://huggingface.co/papers/2602.20159" target="blank"> <img alt="arXiv" src="https://img.shields.io/badge/arXiv-VBVR-red?logo=arxiv" height="20" /> </a> <a href="https://huggingface.co/Video-Reason/VBVR-Wan2.2" target="blank"> <img alt="Leaderboard" src="https://img.shields.io/badge/%F0%9F%A4%97%20VBVRWan2.2-Model-ffc107?color=ffc107&logoColor=white" height="20" /> </a> <a href="https://huggingface.co/datasets/Video-Reason/VBVR-Dataset" target="blank"> <img alt="Dataset" src="https://img.shields.io/badge/%F0%9F%A4%97%20VBVRDataset-Data-ffc107?color=ffc107&logoColor=white" height="20" /> </a> <a href="https://huggingface.co/datasets/Video-Reason/VBVR-Bench-Data" target="blank"> <img alt="Bench Data" src="https://img.shields.io/badge/%F0%9F%A4%97%20VBVRBench-Data-ffc107?color=ffc107&logoColor=white" height="20" /> </a> <a href="https://huggingface.co/spaces/Video-Reason/VBVR-Bench-Leaderboard" target="blank"> <img alt="Leaderboard" src="https://img.shields.io/badge/%F0%9F%A4%97%20VBVR_Bench-Leaderboard-ffc107?color=ffc107&logoColor=white" height="20" /> </a>

Video-MCP is a synthetic video dataset for training and evaluating video generation models on multiple-choice question-answering (MCQA) tasks. Each sample is a short video clip (~5 seconds) where a visual question-answering prompt is embedded directly into the video frames, and the correct answer is revealed by progressively highlighting one of four answer boxes (A/B/C/D) over the duration of the clip.

The dataset is designed for fine-tuning image-to-video models (specifically Wan2.2-I2V-A14B) to produce videos that "answer" visual questions by highlighting the correct option.

Output follows the [VBVR DataFactory](https://github.com/video-reason/VBVR-DataFactory) directory convention.

Examples

Each clip starts with no answer highlighted, then progressively reveals the correct choice over ~5 seconds:

CoreCognition (M-1) — General Visual Reasoning

Answer: BAnswer: B
[image][image]

ScienceQA (M-2) — Science Education

Answer: AAnswer: A
[image][image]

MathVision (M-3) — Competition Math

Answer: AAnswer: D
[image][image]

PhyX (M-4) — Physics Reasoning

Answer: CAnswer: C
[image][image]

Dataset Details

PropertyValue
Version1.0
Total samples6,912
Video resolution832x480
Frame count81 frames per clip
Frame rate16 FPS
Duration~5.06 seconds per clip
CodecH.264, yuv420p, MP4 container
Highlight styledarken (default)

Source Datasets

Video-MCP draws from four publicly available MCQA-VQA datasets on Hugging Face:

Generator IDNameSourceSamplesDomain
M-1corecognitionwilliamium/CoreCognition753General visual reasoning
M-2scienceqaderek-thomas/ScienceQA3,905Science education (image-only subset)
M-3mathvisionMathLLMs/MathVision1,254Competition math with diagrams
M-4phyxCloudriver/PhyX1,000Physics reasoning

All source datasets are filtered to include only samples that have an associated image and exactly four answer choices (A/B/C/D).

Data Structure

Each sample follows the VBVR DataFactory directory convention:

{generator_id}_{name}_data-generator/
  clip_config.json
  {name}_task/
    {name}_{NNNN}/
      first_frame.png        # Frame 0: question visible, no highlight
      prompt.txt             # Plain-text question, choices, and answer
      final_frame.png        # Last frame: correct answer fully highlighted
      ground_truth.mp4       # Full clip with progressive answer reveal
      original/
        question.json        # Structured metadata (JSON)
        <source_image>       # Original image from source dataset

File Descriptions

FileDescription
first_frame.pngThe opening frame showing the question panel (image + question text + four choices) with A/B/C/D answer boxes in the corners. No answer is highlighted.
final_frame.pngThe closing frame with the correct answer box fully highlighted.
ground_truth.mp4The complete video clip. The correct answer gradually highlights from frame 1 to the final frame (linear fade-in).
prompt.txtHuman-readable text: question, choices (A/B/C/D), and the correct answer letter.
original/question.jsonStructured JSON with fields: dataset, source_id, question, choices, answer, original_image_filename.
original/<image>The raw source image preserved with its original filename.
clip_config.jsonGenerator-level config: fps, seconds, num_frames, width, height.

Frame Layout

Each frame uses a two-column layout:

  • —Left column: the source VQA image, scaled to fill.
  • —Right column: question text and the four answer options.
  • —Corners: A (top-left), B (top-right), C (bottom-left), D (bottom-right) answer boxes.

prompt.txt Format

What color is the object in the image?

A: Red
B: Blue
C: Green
D: Yellow

Answer: A

Video Specifications

These defaults align with Wan2.2-I2V-A14B fine-tuning constraints:

  • —Resolution: 832x480 (width and height divisible by 8 for VAE spatial compression)
  • —Frames: 81 (satisfies 1 + 4k for VAE temporal grid)
  • —FPS: 16
  • —Duration: ~5.06 seconds
  • —Codec: H.264, yuv420p pixel format

Intended Use

  • —Fine-tuning image-to-video generation models to produce MCQA-answering videos
  • —Evaluating video generation models on structured visual reasoning tasks
  • —Research on embedding structured UI interactions into generated video

Limitations

  • —All source questions are filtered to exactly 4 choices (A/B/C/D); questions with fewer or more options are excluded.
  • —The answer highlight is a simple linear fade-in; no complex visual dynamics.
  • —Source images and questions inherit any biases or errors from the upstream HF datasets.
  • —The dataset uses a single fixed resolution (832x480) and frame count (81).

Citation

If you use this dataset, please cite the source datasets:

  • —CoreCognition: williamium/CoreCognition on Hugging Face
  • —ScienceQA: Lu et al., "Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering" (NeurIPS 2022)
  • —MathVision: Wang et al., "MathVision: Measuring Multimodal Mathematical Reasoning with Benchmarks" (2024)
  • —PhyX: Cloudriver/PhyX on Hugging Face

License

This dataset is a derivative work. Each source dataset has its own license terms. Users should verify compliance with upstream licenses before redistribution.

Generation Code

https://github.com/video-reason/video-mcp

Video-Reason/video-mcp · CoolFace