Datasetpublic

farrag402/charades-siglip-tokens

Charades SigLIP Token Cache (Stage 1) This is derived data (SigLIP embeddings of video frames), not a redistribution of the source dataset's raw video. Source: Charades (Allen Institute for AI), via the community mirror jinyoungkim/Charades; ~30 s average indoor activity videos. Subset: 5000 videos, stratified over the 157 Charades action classes (multi-label) with fixed seed 42 (exact IDs in manifest.json). Processing: google/siglip-so400m-patch14-384, fully frozen… See the full description on the dataset page: https://huggingface.co/datasets/farrag402/charades-siglip-tokens.

sourceHugging Faceupdated 2mo agoView on Hugging Face

0likes109downloads

Dataset Card

Charades SigLIP Token Cache (Stage 1)

*This is derived data (SigLIP embeddings of video frames), not a redistribution of the source dataset's raw video.*

—Source: Charades (Allen Institute for AI), via the community mirror `jinyoungkim/Charades`; ~30 s average indoor activity videos.
—Subset: 5000 videos, stratified over the 157 Charades action classes (multi-label) with fixed seed 42 (exact IDs in manifest.json).
—Processing: google/siglip-so400m-patch14-384, fully frozen, penultimate transformer layer (final layer deleted, post_layernorm bypassed), 2×2-style bilinear pooling 729 → 196 tokens/frame, 1 fps sampling (capped frames spread uniformly over the full video). No temporal chunking — the flat token sequence carries per-frame timestamps; windowing is left to the consumer.
—File format: one .pt per video containing {"tokens": float16 [T, 196, 1152], "metadata": {...}} — metadata carries row-aligned frame indices/timestamps. The .json files mirror the metadata for quick inspection only; the .pt is the source of truth.
—Intended use: Stage 1 self-supervised pretraining targets for a video compressor project. This is not a general-purpose mirror of the source dataset; captions/QA annotations were deliberately not processed.