datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
OpenVid-1M
Summary
This is the dataset proposed in our paper [ICLR 2025] OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation.
OpenVid-1M is a high-quality text-to-video dataset designed for research institutions to enhance video quality, featuring high aesthetics, clarity, and resolution. It can be used for direct training or as a quality tuning complement to other video datasets.
All videos in the OpenVid-1M dataset have resolutions of at least 512×512.… See the full description on the dataset page: https://huggingface.co/datasets/nkp37/OpenVid-1M.OpenVid-1M-wds
OpenVid-1M — WebDataset repackaging
This repository is a sequential-read-optimized WebDataset repackaging of nkp37/OpenVid-1M by Nan et al. (ICLR 2025). The video content is identical to the original — only the on-disk layout is changed so it can be streamed efficiently from a single HTTP/NFS connection.
What differs from the original
Aspect
Original nkp37/OpenVid-1M
This repository
Format
Per-video mp4 files zipped
WebDataset .tar shards (~2 GB each)… See the full description on the dataset page: https://huggingface.co/datasets/Dev-Jahn/OpenVid-1M-wds.Youtube-Common-First-600-ParquetOpenvid-1M
OpenVid Dataset (Lance Format)
Lance format version of the OpenVid dataset with 937,957 high-quality videos stored with inline video blobs, embeddings, and rich metadata.
Why Lance?
Lance is an open-source format designed for multimodal AI data, offering significant advantages over traditional formats for modern AI workloads.
Blazing Fast Random Access: Optimized for fetching scattered rows, making it ideal for random sampling, real-time ML serving, and interactive… See the full description on the dataset page: https://huggingface.co/datasets/lance-format/Openvid-1M.pexel-0808-complete-final-testGithub Page: https://github.com/UmiMarch/OpenVideo
license: cc-by-4.0
task_categories:
- video-text-to-text
size_categories:
- 100K<n<1M
openvid-lance
OpenVid (Lance Format)
A Lance-formatted version of the OpenVid-1M corpus — 937,957 high-quality clips with inline MP4 bytes, 1024-dim video embeddings, captions, and rich per-clip quality signals — available directly from the Hub at hf://datasets/lance-format/openvid-lance/data/train.lance.
Key features
Inline MP4 bytes in the video_blob column, stored in a side blob file and surfaced as lazy BlobFile handles via take_blobs — metadata scans, search, and filtering… See the full description on the dataset page: https://huggingface.co/datasets/lance-format/openvid-lance.cosmos-openvid-1m
Cosmos-Tokenized OpenVid-1M
Cosmos-Tokenized OpenVid-1M
How to use
Shards are stored in parquet format.
It has 4 columns: serialized_latent, caption, fps, video.
serialized_latent is the latent vector of the video, serialized using torch.save().
Please use the following function to deserialize it:def deserialize_tensor(
serialized_tensor: bytes, device: Optional[str] = None
) -> torch.Tensor:
return torch.load(
io.BytesIO(serialized_tensor)… See the full description on the dataset page: https://huggingface.co/datasets/fal/cosmos-openvid-1m.OpenVidHD-0.4M-720p-48fpsopen-vi-dialog-500h
OpenVI Dialog Synthetic 500h
Vietnamese two-speaker synthetic dialogue data for ZipVoice-Dialog experiments.
60,000 audio chunks of 30 seconds each
500 hours total
Five shards of approximately 100 hours
FLAC audio with per-chunk dialogue manifests and turn metadata
Generated with the OpenDialog v2 synthesis pipeline
Each shard contains a manifest.jsonl and its corresponding chunks/ directory.
open_video_dataOpenVid-1M
Summary
This is the dataset proposed in our paper "OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation".
OpenVid-1M is a high-quality text-to-video dataset designed for research institutions to enhance video quality, featuring high aesthetics, clarity, and resolution. It can be used for direct training or as a quality tuning complement to other video datasets.
All videos in the OpenVid-1M dataset have resolutions of at least 512×512. Furthermore, we… See the full description on the dataset page: https://huggingface.co/datasets/lodestone-horizon/OpenVid-1M.OpenVid-60k-split
Combination of part_id's from bigdata-pw/OpenVid-1M and video data from nkp37/OpenVid-1M.
This is a 60k video split of the original dataset for faster iteration during testing. The split was obtained by filtering on aesthetic and motion scores by iteratively increasing their values until there were at most 1000 videos. Only videos containing between 80 and 240 frames were considered.
from datasets import load_dataset, disable_caching, DownloadMode
from torchcodec.decoders import… See the full description on the dataset page: https://huggingface.co/datasets/finetrainers/OpenVid-60k-split.OpenVid-10k-split
Combination of part_id's from bigdata-pw/OpenVid-1M and video data from nkp37/OpenVid-1M.
This is a 10k video split of the original dataset for faster iteration during testing. The split was obtained by filtering on aesthetic and motion scores by iteratively increasing their values until there were at most 1000 videos. Only videos containing between 80 and 240 frames were considered.
from datasets import load_dataset, disable_caching, DownloadMode
from torchcodec.decoders import… See the full description on the dataset page: https://huggingface.co/datasets/finetrainers/OpenVid-10k-split.14K_OpenVid-1MThis is a 14K mp4 and txt pair distillation of the large 1M Openvid dataset. It's filtered to only have 720p and above videos.
Originally created to train a video control LoRA, but can be used for any purpose.
Original dataset: https://huggingface.co/datasets/nkp37/OpenVid-1M
License is the same as the original repo:
Our OpenVid-1M is released as CC-BY-4.0. The video samples are collected from publicly available datasets. Users must follow the related licenses Panda, ChronoMagic… See the full description on the dataset page: https://huggingface.co/datasets/jqlive/14K_OpenVid-1M.OpenVid-1M-mapping
Summary
This is the extent dataset proposed in the paper "OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation".
OpenVid-1M is a high-quality text-to-video dataset designed for research institutions to enhance video quality, featuring high aesthetics, clarity, and resolution. It can be used for direct training or as a quality tuning complement to other video datasets.
New Feature: Video-ZIP mapping files now available for efficient video lookup (see Dataset… See the full description on the dataset page: https://huggingface.co/datasets/phil329/OpenVid-1M-mapping.OpenVid-1M-webshart-indices
OpenVid-1M webshart indices
Webshart indices for Dev-Jahn/OpenVid-1M-wds.
Each JSON file contains byte offsets and sample metadata for the correspondingly named uncompressed tar shard. Paths mirror the source dataset's train/ and val/ directories. The indices are generated remotely with webshart==0.4.6; progress.json records the pinned source revision and upload progress.
Use with webshart
Install webshart:
pip install -U webshart
Pass this repository as the… See the full description on the dataset page: https://huggingface.co/datasets/webshart/OpenVid-1M-webshart-indices.openvid-wantrack-processed
OpenVid-WanTrack — preprocessed training data
Preprocessed (FastVideo parquet) data used to train the TrackWan point-track-conditioned
video model. Derived from OpenVid; each row is one 121-frame clip with everything the
trainer consumes, so it is self-contained (no re-encoding needed).
Contents (parquet columns)
Each field is stored as *_bytes / *_shape / *_dtype (numpy buffers):
vae_latent — Wan VAE latent of the clip
first_frame_latent — I2V conditioning… See the full description on the dataset page: https://huggingface.co/datasets/noctuashap/openvid-wantrack-processed.OpenVideo-Scene-Reasoning
OpenVideo-Scene-Reasoning
OpenVideo-Scene-Reasoning is a video understanding dataset containing 2,894 short video clips, where each sample consists of a 10-second video, five uniformly sampled frames, and a dense scene-level response describing the complete temporal sequence.
Rather than generating captions for individual frames independently, the responses are synthesized by jointly reasoning over the sampled frames to capture temporal progression, object interactions, actions… See the full description on the dataset page: https://huggingface.co/datasets/prithivMLmods/OpenVideo-Scene-Reasoning.openvid-1m-subsetopenvid-wantrack-processed-v2
OpenVid-WanTrack Processed (v2, 720p, bf16)
FastVideo preprocessing parquets for training the TrackWan point-track-conditioned I2V model on
the OpenVid-derived WanTrack set. Each row is one 121-frame clip with its VAE latents, text and
image conditioning, and dense CoTracker3 tracks — everything the trainer memory-maps, so no video
decoding happens at train time.
This is the bfloat16 variant of …/openvid-wantrack-processed (v2, 720p): the large float
tensor fields are stored in… See the full description on the dataset page: https://huggingface.co/datasets/FastVideo/openvid-wantrack-processed-v2.openvid-wantrack-tracks
OpenVid-WanTrack — 121-frame tracks (tar shards)
121-frame filtered tracks from OpenVid used to build the TrackWan training set (~259k npzs, ~666 GB).
Format: packed into tracks-XXXXX.tar shards (~1,000 npz per shard) for efficient hosting.
Each file is <video_id>_<clip>_<start>to<end>.npz. The basename joins 1:1 with the
tracks and processed datasets. Consume by streaming the tars (WebDataset) or tar -xf.
Apache-2.0; derived from OpenVid (respect upstream terms).
openvid-hftv-gops
OpenVid GOP cache for HFTV
Fixed-size one-second video clips, packed for training a spatiotemporal
autoencoder. Derived from OpenVid-1M
(Nan et al., OpenVid-1M: A Large-Scale High-Quality Dataset for
Text-to-video Generation, ICLR 2025), used and redistributed under
CC BY-4.0; this derivative carries the same licence.
Why it exists
OpenVid-1M ships as 12.4 TB of ~50 GB zip parts, so selecting clips by
caption cannot reduce what you fetch -- only what you keep.… See the full description on the dataset page: https://huggingface.co/datasets/arodland/openvid-hftv-gops.OpenVid384pxopen-vi-dialog-synthetic-100h
OpenDialog Vietnamese Synthetic Dialogue 100h
Synthetic Vietnamese two-speaker dialogue for ZipVoice-Dialog experiments.
12,000 chunks
30 seconds per chunk
100.0 hours total
Each item contains S1/S2 speaker labels, turn timings, target text,
relationship, pronouns, environment, topic, mood, and source reference IDs.
Audio renderer: vLLM-Omni VoxCPM2
Audio format: mono WAV, 48 kHz, 30 seconds per chunk
This is a research dataset. Review the source/reference licensing and the… See the full description on the dataset page: https://huggingface.co/datasets/tsdocode/open-vi-dialog-synthetic-100h.prompttest-18videosYoutube-Common-First-600YouTube-Commons-5G-RawOpenVid-1k-split
Combination of part_id's from bigdata-pw/OpenVid-1M and video data from nkp37/OpenVid-1M.
This is a 1k video split of the original dataset for faster iteration during testing. The split was obtained by filtering on aesthetic and motion scores by iteratively increasing their values until there were at most 1000 videos. Only videos containing between 80 and 240 frames were considered.
Loading the data:
from datasets import load_dataset, disable_caching, DownloadMode
from… See the full description on the dataset page: https://huggingface.co/datasets/finetrainers/OpenVid-1k-split.openvid-frame-sequences-1M
OpenVid Frame Sequences — 1M adjacent frame pairs
Short, single-shot frame sequences cut from OpenVid-1M,
built to train and evaluate models on what changes between two frames half a second apart.
One sample = 10 consecutive frames, 0.5 s apart (a 4.5 s span) → 9 adjacent frame pairs.
[f00] --0.5s--> [f01] --0.5s--> [f02] ... [f09]
^ the thing you describe / predict
Sequences
116,596
Frames per sequence
10 (0.5 s apart, t = 0.0 … 4.5 s)
Adjacent frame… See the full description on the dataset page: https://huggingface.co/datasets/junha1125/openvid-frame-sequences-1M.OpenVID-Cotracker每个生成的 .npz 文件包含以下两个数组：
tracks :
含义 ：点的运动轨迹坐标。
形状 (Shape) : (1, T, N, 2)
1 : Batch size (默认为 1)。
T : 视频的总帧数。
N : 采样点的数量（这里是30）。
2 : 每个点的坐标 (x, y) 。
visibility :
含义 ：点的可见性掩码。
形状 (Shape) : (1, T, N)
1 : Batch size。
T : 视频的总帧数。
N : 采样点的数量。
数值 ：通常为布尔值或 0/1，代表该点在当前帧是否被遮挡或移出画面。