datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
LongVideoBench
Dataset Card for LongVideoBench
Large multimodal models (LMMs) are handling increasingly longer and more complex inputs. However, few public benchmarks are available to assess these advancements. To address this, we introduce LongVideoBench, a question-answering benchmark with video-language interleaved inputs up to an hour long. It comprises 3,763 web-collected videos with subtitles across diverse themes, designed to evaluate LMMs on long-term multimodal understanding.
The… See the full description on the dataset page: https://huggingface.co/datasets/longvideobench/LongVideoBench.LongVideoBench
Dataset Card for LongVideoBench
Large multimodal models (LMMs) are handling increasingly longer and more complex inputs. However, few public benchmarks are available to assess these advancements. To address this, we introduce LongVideoBench, a question-answering benchmark with video-language interleaved inputs up to an hour long. It comprises 3,763 web-collected videos with subtitles across diverse themes, designed to evaluate LMMs on long-term multimodal understanding.
The… See the full description on the dataset page: https://huggingface.co/datasets/lccshunli/LongVideoBench.LongVideoBenchLong-video-test-datalongvideo_eval_videos
Long-RL: Scaling RL to Long Sequences (Evaluation Dataset - for research only)
Data Distribution
We strategically construct a high-quality dataset with CoT annotations for long video reasoning, named LongVideo-Reason. Leveraging a powerful VLM (NVILA-8B) and a leading open-source reasoning LLM, we develop a dataset comprising 52K high-quality Question-Reasoning-Answer pairs for long videos. We use 18K high-quality samples for Long-CoT-SFT to initialize… See the full description on the dataset page: https://huggingface.co/datasets/LongVideo-Reason/longvideo_eval_videos.LongVT-Parquet
LongVT-Parquet
This repository contains the training data annotations and evaluation benchmark for the LongVT project.
Overview
LongVT is an end-to-end agentic framework that enables "Thinking with Long Videos" via interleaved Multimodal Chain-of-Tool-Thought. This dataset provides the training annotations and evaluation benchmark in Parquet format, with source media files available in LongVT-Source.
Important Notes
For privacy reasons, media paths… See the full description on the dataset page: https://huggingface.co/datasets/longvideotool/LongVT-Parquet.LongVideoHaystack
LV-Haystack: Temporal Search for Long-Form Video Understanding
Jinhui Ye1*,
Zihan Wang2*,
Haosen Sun2,
Keshigeyan Chandrasegaran1,
Zane Durante1,
Cristobal Eyzaguirre1,
Yonatan Bisk3,
Juan Carlos Niebles1,
Ehsan Adeli1,
Li Fei-Fei1,
Jiajun Wu1,
Manling Li2
Stanford University1, Northwestern University2, Carnegie Mellon University3
CVPR 2025 . Dataset is part of the T* project
🌎Website |
🧑💻Code |
📄arXiv |
🏆 Leaderboard (Coming Soon)… See the full description on the dataset page: https://huggingface.co/datasets/MLL-Lab/LongVideoHaystack.VideoSIAH-Eval
VideoSIAH-Eval
Evaluation benchmark for LongVT, containing 652 unique QA pairs across 244 long-form videos with human-in-the-loop validation.
Update (2026-03): The initial release contained 1,280 entries due to unintentional duplication during data export. This version has been cleaned to 652 unique QA pairs. Since each entry was an exact copy, all evaluation metrics reported in the paper remain unchanged.
longvideo-reason
Long-RL: Scaling RL to Long Sequences (Training, Validation and Test Dataset - for research only)
Data Distribution
We strategically construct a high-quality dataset with CoT annotations for long video reasoning, named LongVideo-Reason. Leveraging a powerful VLM (NVILA-8B) and a leading open-source reasoning LLM, we develop a dataset comprising 52K high-quality Question-Reasoning-Answer pairs for long videos. We use 18K high-quality samples for… See the full description on the dataset page: https://huggingface.co/datasets/LongVideo-Reason/longvideo-reason.LongVideoBench-MetaBenchCheck-LongVideo
BenchCheck-LongVideo: frame-budget ladder for three open models (task 13b run package)
Run package for an agent on a separate GPU machine. Goal: on each of 30 long-video benchmarks
(mean video duration >= 300 s), answer the same up to 300 (155 to 320 multiple-choice items per benchmark, 8481 in total) multiple-choice items with THREE models at
four frame budgets, 32 / 128 / 512 / 1024 frames, at the model's own default resolution, and send
the per-item outputs back. The analysis… See the full description on the dataset page: https://huggingface.co/datasets/GMLRVigil/BenchCheck-LongVideo.LongVideo-Reason-4k-Video-Crop-Handoff-20260911
LongVideo-Reason 4k · Video Crop 合成移交包
公开仓库，文件访问需要人工审批。 只有仓库根目录出现 READY.json 且 complete=true 时，才表示所有 QA、视频、pipeline 和校验信息已齐备；此前为准备／上传阶段。
本包用于将原视频和原始 QA 重新合成为视频工具轨迹。它不是已经审核通过的 SFT 数据，也不把原论文 reasoning 当作工具轨迹监督。
内容
文件
用途
data/qa.jsonl
4,000 条原始 LongVideo-Reason train QA、原选项、原答案和来源
videos/*.mp4
配套原视频；与 QA 的 video_path 对应
data/video_manifest.jsonl
每个视频的 SHA-256、CRC、ffprobe 时长、尺寸和镜像来源
data/selection_report.json
最终数量、时长分布、去重和筛选范围… See the full description on the dataset page: https://huggingface.co/datasets/b1intern/LongVideo-Reason-4k-Video-Crop-Handoff-20260911.LongTVQA
LongTVQA Dataset Files
This repository contains the LongTVQA dataset exports as JSON-formatted files.
Despite the .jsonl suffix, the QA splits are stored as a JSON array, and the
subtitle files are JSON objects.
Data files
LongTVQA_train.jsonl — training split QA list.
LongTVQA_val.jsonl — validation split QA list.
LongTVQA_subtitles_clip_level.jsonl — clip-level subtitle text indexed by
occur_clip (e.g. castle_s01e01_seg02_clip_00).… See the full description on the dataset page: https://huggingface.co/datasets/longvideoagent/LongTVQA.LongTVQA_plus
LongTVQA+ Dataset
This repository contains the LongTVQA+ dataset in JSON format.
LongTVQA+ is built upon the original TVQA+ dataset, with the key difference that it extends the question grounding scope from short clip-level segments (≈1 minute) to long episode-level videos (up to ~20 minutes).This enables research on long-form video understanding, long-range temporal reasoning, and fine-grained spatio-temporal grounding in realistic TV show episodes.
In addition to the extended… See the full description on the dataset page: https://huggingface.co/datasets/longvideoagent/LongTVQA_plus.LongVideoDB-373K-IterCaplongvideoslongvideoreason_filterLongVideoHaystack_tiny
Dataset Statistics Summary
Metric
Total
Tiny
Test
Video Statistics
Total Videos
Total Video Duration (hr)
Avg. Video Duration (min)
Clip Statistics
Total Video Clips
Total Video Clip Duration (hr)
Avg. Video Clip Duration (sec)
Frame Statistics
Total Frames (k)
Avg. Frames per Video (k)
Ratio of Keyframe / Frame (‰)
QA Statistics
Total QA Pairs
Avg. QA Pair per Video
Avg. QA Pair per Clip
Avg.… See the full description on the dataset page: https://huggingface.co/datasets/LVHaystack/LongVideoHaystack_tiny.LongVideoBench-Longvrlm-longvideobench-sft
Video RLM Trajectory SFT
This dataset contains per-turn supervised fine-tuning samples distilled from
multimodal Recursive Language Model trajectories on LongVideoBench.
The standard RLM was extended with the ability to parse through frames/subtitles,
allowing it to process arbitrary lengths of video data without exceeding the context window.
The orchestrator and subcall models were both gpt-5.4-mini. 103 correct trajectories from
evaluating the model on LongVideoBench were… See the full description on the dataset page: https://huggingface.co/datasets/bkal01/vrlm-longvideobench-sft.longvideos2longvideos3Long-Video-Tuning-QAslongvideo-reasonlongvideocaptionsXunGuang-LongVideo-1.1M
XunGuang-LongVideo-1.1M
This repository hosts an anonymously released preview version of the XunGuang-LongVideo-1.1M dataset for the ECCV review process.
Status (Anonymous Preview)
Current release: 100 samples (for demonstration and verification purposes)
Planned full release (after ECCV review): the complete dataset at the 1.1M scale
After the ECCV review process is completed, we will publish the full dataset and provide a more complete documentation page.… See the full description on the dataset page: https://huggingface.co/datasets/annouser4eccv/XunGuang-LongVideo-1.1M.LongVideo-Reason