CoolFace
Datasetpublic

mvp-lab/LLaVA-OneVision-2-Data

LLaVA-OneVision-2-Data Training data for the LLaVA-OneVision-2 multimodal model family. The release contains large-scale video data at several duration ranges, video captions and source mappings, and spatial-reasoning data used for mid-training. At a Glance The dataset is split across two Hugging Face repositories because of its size: Repository What it contains Part 1 (this repository) ~60-second video shards, captions for all duration ranges… See the full description on the dataset page: https://huggingface.co/datasets/mvp-lab/LLaVA-OneVision-2-Data.

sourceHugging Faceapache-2.0updated 23d agoView on Hugging Face
39likes131kdownloads
Dataset Card

LLaVA-OneVision-2-Data

Training data for the LLaVA-OneVision-2 multimodal model family. The release contains large-scale video data at several duration ranges, video captions and source mappings, and spatial-reasoning data used for mid-training.

At a Glance

The dataset is split across two Hugging Face repositories because of its size:

RepositoryWhat it contains
[Part 1](https://huggingface.co/datasets/mvp-lab/LLaVA-OneVision-2-Data) (this repository)~60-second video shards, captions for all duration ranges, source-video mappings, spatial-reasoning shards, and browser previews
[Part 2](https://huggingface.co/datasets/mvp-lab/LLaVA-OneVision-2-Data-Part2)The full ~180-second and >10-minute video shards

If you need the long-video data, download the video shards from Part 2 and use the corresponding captions and mapping files from Part 1.

Data Locations

Part 1: metadata, previews, spatial data, and ~60s videos

DataPathFormatDescription
~60s videosmid_training_video/60s_rest/WebDataset (.tar)10,809 video shards
30s captionsmid_training_video/caption_v0/split_30s.jsonlJSONLCaptions for 30-second clips
60s captionsmid_training_video/caption_v0/split_60s.jsonlJSONLCaptions for 60-second clips
180s captionsmid_training_video/caption_v0/split_180s.jsonlJSONLCaptions for the 180-second videos in Part 2
>10min captionsmid_training_video/caption_v0/split_gt10min.jsonlJSONLCaptions for the >10-minute videos in Part 2
Source mappingsmid_training_video/mapping/mapping_{5s,10s,30s,60s,180s,gt10min}.csvCSVMaps each clip's dst_path to its source youtube_id and [start_time, end_time] window
Spatial reasoningspatial/WebDataset (.tar)84 shards covering refcoco, Visual Genome, pointing, 3D, and related tasks

Part 2: long-video WebDataset shards

The directory names in Part 2 are upload prefixes. Their duration mapping is:

DurationPart 2 directorySample path inside each archiveShards currently presentApprox. size
~180 seconds`tom/`180s/...4,19913.6 TB
>10 minutes`jerry/`gt10min/...21,05270.5 TB

The mapping above was verified from the sample paths stored inside the WebDataset archives.

How the Files Fit Together

For the ~180-second split:

  1. 1.Download video shards from tom/ in Part 2.
  2. 2.Read captions from mid_training_video/caption_v0/split_180s.jsonl in Part 1.
  3. 3.Use mid_training_video/mapping/mapping_180s.csv when the source YouTube ID or time window is needed.

For the >10-minute split:

  1. 1.Download video shards from jerry/ in Part 2.
  2. 2.Read captions from mid_training_video/caption_v0/split_gt10min.jsonl in Part 1.
  3. 3.Use mid_training_video/mapping/mapping_gt10min.csv when the source YouTube ID or time window is needed.

Downloading the Long-Video Shards

Install the Hugging Face CLI, then download only the duration range you need.

bash
# ~180-second videos (4,199 shards, approximately 13.6 TB)
hf download mvp-lab/LLaVA-OneVision-2-Data-Part2 \
  --repo-type dataset \
  --include "tom/**" \
  --local-dir ./LLaVA-OneVision-2-Data-Part2

# >10-minute videos (21,052 shards currently present, approximately 70.5 TB)
hf download mvp-lab/LLaVA-OneVision-2-Data-Part2 \
  --repo-type dataset \
  --include "jerry/**" \
  --local-dir ./LLaVA-OneVision-2-Data-Part2

The long-video data is very large. Use --include to avoid downloading both duration ranges when only one is required.

Dataset Viewer Previews

The viewer_* configs expose small Parquet samples for schema inspection in the Hugging Face Dataset Viewer:

ConfigPreview contents
viewer_caption_30s5 caption samples from 30-second clips
viewer_caption_60s5 caption samples from 60-second clips
viewer_caption_180s3 caption samples from 180-second clips
viewer_caption_gt10min1 caption sample from a >10-minute clip
viewer_spatial10 spatial-reasoning samples with embedded thumbnails

These configs are for preview and schema inspection only. Use the full WebDataset shards for training.

Part 2 Upload Status

The jerry/ filenames follow a train_XXXXX_of_21092.tar naming scheme. At the time of this README update, Part 2 contains 21,052 of those shard files. Check the Part 2 file browser for the latest upload status before starting a full download.

Citation

If you find LLaVA-OneVision-2 useful in your research, please cite the following papers:

bibtex
@misc{an2026llavaonevision2nextgenerationperceptualintelligence,
  title         = {LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence},
  author        = {Xiang An and Yin Xie and Feilong Tang and Yunyao Yan and Huajie Tan and Didi Zhu and Changrui Chen and Xiuwei Zhao and Bin Qin and Kaicheng Yang and Yifei Shen and Yuanhan Zhang and Kaichen Zhang and Wenkang Zhang and Zheng Cheng and Nansen Zhang and Chunsheng Wu and Chunjiang Ge and Zimin Ran and Dehua Song and Chunyuan Li and Shikun Feng and Ming Hu and Zhangquan Chen and Junbo Niu and Bo Li and Ziyong Feng and Ziwei Liu and Zongyuan Ge and Jiankang Deng},
  year          = {2026},
  eprint        = {2605.25979},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2605.25979}
}

@misc{an2025llavaonevision15fullyopenframework,
  title         = {LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Training},
  author        = {Xiang An and Yin Xie and Kaicheng Yang and Wenkang Zhang and Xiuwei Zhao and Zheng Cheng and Yirui Wang and Songcen Xu and Changrui Chen and Didi Zhu and Chunsheng Wu and Huajie Tan and Chunyuan Li and Jing Yang and Jie Yu and Xiyao Wang and Bin Qin and Yumeng Wang and Zizhen Yan and Ziyong Feng and Ziwei Liu and Bo Li and Jiankang Deng},
  year          = {2025},
  eprint        = {2509.23661},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2509.23661}
}

@misc{tang2026onevisionencodercodecalignedsparsityfoundational,
  title         = {OneVision-Encoder: Codec-Aligned Sparsity as a Foundational Principle for Multimodal Intelligence},
  author        = {Feilong Tang and Xiang An and Yunyao Yan and Yin Xie and Bin Qin and Kaicheng Yang and Yifei Shen and Yuanhan Zhang and Chunyuan Li and Shikun Feng and Changrui Chen and Huajie Tan and Ming Hu and Manyuan Zhang and Bo Li and Ziyong Feng and Ziwei Liu and Zongyuan Ge and Jiankang Deng},
  year          = {2026},
  eprint        = {2602.08683},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2602.08683}
}
mvp-lab/LLaVA-OneVision-2-Data · CoolFace