Datasetpublic

oscarqjh/MMSI-Video-Bench_lmmseval

MMSI-Video-Bench A video-based spatial intelligence benchmark for evaluating Multimodal Large Language Models (MLLMs). Dataset Description MMSI-Video-Bench tests models on: Spatial reasoning Motion understanding Planning and prediction Cross-video reasoning Dataset Structure MMSI-Video-Bench/ ├── data/ │ └── test-00000-of-00001.parquet # 1106 samples ├── frames.zip # Extracted video frames ├── ref_images.zip… See the full description on the dataset page: https://huggingface.co/datasets/oscarqjh/MMSI-Video-Bench_lmmseval.

sourceHugging Facecc-by-4.0updated 8mo agoView on Hugging Face

0likes107downloads

Dataset Card

MMSI-Video-Bench

A video-based spatial intelligence benchmark for evaluating Multimodal Large Language Models (MLLMs).

Dataset Description

MMSI-Video-Bench tests models on:

—Spatial reasoning
—Motion understanding
—Planning and prediction
—Cross-video reasoning

Dataset Structure

MMSI-Video-Bench/
├── data/
│   └── test-00000-of-00001.parquet   # 1106 samples
├── frames.zip                         # Extracted video frames
├── ref_images.zip                     # Reference images
├── videos.zip                         # Original videos
└── README.md

Parquet Columns

Column	Type	Description
`id`	string	Unique question ID (e.g., "question_0000")
`type`	string	Question category
`system_prompt`	string	System prompt
`task_prompt`	string	Task description
`user_prompt`	string	User question
`format_prompt`	string	Answer format instructions
`ori_question`	string	Original question text
`options`	list[string]	Answer choices (A-F)
`ground_truth`	string	Correct answer letter
`video_list`	list[dict]	Video metadata (path, start, end, base_fps)
`frames_list`	list[list[string]]	Paths to extracted frames per video
`ref_images`	list[string]	Paths to reference images

Usage

python

import pandas as pd
from datasets import load_dataset

# Option 1: Load parquet directly
df = pd.read_parquet("hf://datasets/oscarqjh/MMSI-Video-Bench/data/test-00000-of-00001.parquet")

# Option 2: Load with datasets library
dataset = load_dataset("oscarqjh/MMSI-Video-Bench")

# Access a sample
sample = dataset["train"][0]
print(sample["id"])
print(sample["ori_question"])
print(sample["options"])
print(sample["ground_truth"])

# Video paths are relative to the extracted zip files
# e.g., sample["video_list"][0]["path"] = "question_0000/video.mp4"
# After extracting videos.zip: videos/question_0000/video.mp4

Extracting Media Files

bash

# Download and extract
wget https://huggingface.co/datasets/oscarqjh/MMSI-Video-Bench/resolve/main/frames.zip
wget https://huggingface.co/datasets/oscarqjh/MMSI-Video-Bench/resolve/main/ref_images.zip
wget https://huggingface.co/datasets/oscarqjh/MMSI-Video-Bench/resolve/main/videos.zip

unzip frames.zip -d frames/
unzip ref_images.zip -d ref_images/
unzip videos.zip -d videos/

Citation

Please cite the original MMSI-Video-Bench paper if you use this dataset.

License

CC-BY-4.0