TornadoLabs/howto100m
HowTo100M 105,128 videos (11721.4 GB) with 104,584 subtitle files, downloaded at source quality and re-hosted for direct use — no more dead YouTube links, no more flaky downloader scripts. Coverage: 105,128 of the 1,238,911 source video IDs (8.5%). 19,160 source videos were unavailable at fetch time (private, removed, members-only or geo-blocked) and are excluded. The dataset is refreshed as more videos are delivered. What's inside metadata.jsonl — one row per… See the full description on the dataset page: https://huggingface.co/datasets/TornadoLabs/howto100m.
HowTo100M
105,128 videos (11721.4 GB) with 104,584 subtitle files, downloaded at source quality and re-hosted for direct use — no more dead YouTube links, no more flaky downloader scripts.
Coverage: 105,128 of the 1,238,911 source video IDs (8.5%). 19,160 source videos were unavailable at fetch time (private, removed, members-only or geo-blocked) and are excluded. The dataset is refreshed as more videos are delivered.
What's inside
- `metadata.jsonl` — one row per video:
video_id,video_url(direct HTTPS download, CDN-backed),subtitle_url,youtube_url,size_bytes - The 105,128 MP4s (11721.4 GB) and 104,584 subtitle files are hosted on Hugging Face storage and served via the
video_urlcolumn - `captions.parquet` — 1,238,911 academic annotations (source:
annotations.csv)
Quick start
from datasets import load_dataset
# videos (default config): one row per video with its direct download URL
ds = load_dataset("TornadoLabs/howto100m", "videos", split="train")
print(ds[0])
# download one video
import requests
r = requests.get(ds[0]["video_url"])
open(f"{ds[0]['video_id']}.mp4", "wb").write(r.content)
# academic annotations
ann = load_dataset("TornadoLabs/howto100m", "annotations", split="train")How this dataset was built
Every video was fetched through the Tornado API bulk pipeline and delivered straight into Hugging Face storage via its S3-compatible gateway — the exact workflow documented here.
Of the original ID list, only videos still publicly available on YouTube are included (dead/private videos are listed upstream and excluded here).
Provenance & citation
- Original dataset: https://www.di.ens.fr/willow/research/howto100m/
- Paper: Miech et al., ICCV 2019 — HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips
- 1.2M vidéos instructionnelles.
The video files remain the property of their respective YouTube uploaders. The annotations belong to the original dataset authors — cite their paper if you use them. This re-hosting is provided for reproducible research; if you are a rights holder and want content removed, open a discussion on this repo.
Build your own video dataset
Need a custom corpus (different IDs, other platforms, higher volume)? [Tornado API](https://tornadoapi.io?ref=hf-howto100m-cta) downloads YouTube at scale — bulk endpoints, S3/R2/HF delivery, subtitles and metadata included.
