Datasetpublic

quastAI/behavior-1k-2025-challenge-vjepa2-vitg-demo-embeddings

V-JEPA 2 ViT-G Embeddings — BEHAVIOR-1K 2025 Challenge Demos (62h) Precomputed video embeddings for a 62-hour subsample of the BEHAVIOR-1K 2025 challenge demonstrations, extracted with the V-JEPA 2 ViT-g encoder. The goal is to make downstream experimentation faster and more reproducible by eliminating repeated video decoding and encoder forward passes — lowering the barrier for teams without access to large GPU clusters. Field Value Source dataset… See the full description on the dataset page: https://huggingface.co/datasets/quastAI/behavior-1k-2025-challenge-vjepa2-vitg-demo-embeddings.

sourceHugging Facemitupdated 4mo agoView on Hugging Face

2likes1.2kdownloads

Dataset Card

V-JEPA 2 ViT-G Embeddings — BEHAVIOR-1K 2025 Challenge Demos (62h)

Precomputed video embeddings for a 62-hour subsample of the BEHAVIOR-1K 2025 challenge demonstrations, extracted with the V-JEPA 2 ViT-g encoder. The goal is to make downstream experimentation faster and more reproducible by eliminating repeated video decoding and encoder forward passes — lowering the barrier for teams without access to large GPU clusters.

Field	Value
Source dataset	`behavior-1k/2025-challenge-demos`
Subset size	~62 hours
Rows	~1.1M
Encoder	V-JEPA 2 ViT-g
Representation	Precomputed video embeddings + proprioception + actions
Domain	Embodied AI · Robotics · World Models
License	MIT

Note: This repository contains precomputed feature representations only. It does not redistribute V-JEPA 2 source code or model weights.

Dataset Schema

Each row corresponds to one subsampled step/frame + actions/propios at 5 FPS from the original 30 FPS episodes.

Token columns — visual embeddings (one per camera view)

Column	Type	Shape	Notes
`tokens_head`	`float16 ndarray`	`(256, 1408)`	Head camera
`tokens_left_wrist`	`float16 ndarray`	`(256, 1408)`	Left wrist camera
`tokens_right_wrist`	`float16 ndarray`	`(256, 1408)`	Right wrist camera

256 spatial patches = (256px / 16px patch)² with tubelet size 2. 1408 = ViT-G embedding dim. float16 — bfloat16 is cast down at encoding time.

Proprioceptive columns — one entry per sampled frame

Column	Type	Shape	Notes
`actions`	`float32 ndarray`	`(138,)`	Flattened action chunk from frame f to f+1; `fstp=6` at 30fps→5fps, 23 DoF → shape `(6×23,)`
`states`	`float32 ndarray`	`(133,)`	Curated proprioceptive state at frame f
`cam_rel_poses`	`float32 ndarray`	`(21,)`	3 cameras × (position `[3]` + quaternion `[4]`) at frame f

Index columns — scalars

Column	Type	Description
`frame_index`	`int`	Source MP4 frame number
`episode_idx`	`int`	Episode index within the dataset
`sample_idx`	`int`	Manifest index for the episode
`step_pos`	`int`	0-indexed position of this row within its episode
`episode_len`	`int`	Total rows in this episode

Dataset Pipeline

For details on dataset creation and processing, see the SHARP-Laps pipeline repository.

Licensing

This dataset is released under the MIT license.

—Source demonstrations: `behavior-1k/2025-challenge-demos` — MIT
—V-JEPA 2 (Meta / FAIR): facebookresearch/vjepa2 — Apache-2.0 (not redistributed)

Users should comply with the licenses of both the original BEHAVIOR-1K dataset and the V-JEPA 2 resources.

Citation

If you use this dataset, please cite BEHAVIOR-1K and V-JEPA 2 and V JEPA 2.1.

bibtex

@misc{Quast2026,
  title={Short Horizon Planning with V-JEPA-2 AC on BEHAVIOR-1K},
  author={Quast, Julian},
  year={2026},
}

@article{li2024behavior,
  title={Behavior-1k: A human-centered, embodied ai benchmark with 1,000 everyday activities and realistic simulation},
  author={Li, Chengshu and Zhang, Ruohan and Wong, Josiah and ...},
  journal={arXiv preprint arXiv:2403.09227},
  year={2024}
}

@article{assran2025vjepa2,
  title={V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning},
  author={Assran, Mahmoud and Bardes, Adrien and Fan, David and ...},
  journal={arXiv preprint arXiv:2506.09985},
  year={2025}
}

@article{murlabadia2026vjepa2_1,
  title={V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learning},
  author={Mur-Labadia, Lorenzo and Muckley, Matthew and Bar, Amir and ...},
  journal={arXiv preprint arXiv:2603.14482},
  year={2026}
}