CoolFace
Datasetpublic

Rithvik762/VLNCE-EnvDrop

VLNCE-EnvDrop Synthetic Vision-Language Navigation (VLN) data-augmentation set, derived from the EnvDrop augmentation used in VLN-CE / NaVILA-style training. Each of the 146,304 samples pairs a short first-person navigation video with the natural-language instruction the agent was following and the discrete action sequence it executed. This dataset provides the visual + motion supervision for training a GRU-augmented Qwen3-VL navigation model: the language conditions the… See the full description on the dataset page: https://huggingface.co/datasets/Rithvik762/VLNCE-EnvDrop.

sourceHugging Facemitupdated 2mo agoView on Hugging Face
0likes121downloads
Dataset Card

VLNCE-EnvDrop

Synthetic Vision-Language Navigation (VLN) data-augmentation set, derived from the EnvDrop augmentation used in VLN-CE / NaVILA-style training. Each of the 146,304 samples pairs a short first-person navigation video with the natural-language instruction the agent was following and the discrete action sequence it executed.

This dataset provides the visual + motion supervision for training a GRU-augmented Qwen3-VL navigation model: the language conditions the backbone, while the per-step motion sequence feeds a GRU whose output is projected into the LLM embedding space.

Contents

FileSizeWhat it is
envdrop_videos_00.tar … envdrop_videos_14.tar~270 GBThe raw first-person navigation videos, one <video_id>.mp4 per sample, sharded into 15 tarballs.
envdrop_motion.json392 MBPrimary training annotation. One record per sample: instruction, decoded frame paths, and the per-step action (motion) sequence.
annotations.json22.5 MBLightweight video_id → instruction index (powers the dataset preview). A subset of the info in envdrop_motion.json.

Record schema — envdrop_motion.json

json
{
  "video_id": "34300",
  "q": "Walk forward and stop at the end of the aisle.",
  "frames": ["34300/frame_0.jpg", "34300/frame_1.jpg", "..."],
  "motion": [3, 3, 1, 1, 3, 1, 1, 2, 2, 1, "..."]
}
  • —`video_id` — key into the tarballs (<video_id>.mp4).
  • —`q` — the natural-language navigation instruction.
  • —`frames` — decoded frame paths for the clip (frames are extracted from the corresponding .mp4 at load time; they are not stored separately).
  • —`motion` — the discrete action taken at each step (small action vocabulary, e.g. forward / turn-left / turn-right / stop). This is the GRU input.

Layout

VLNCE-EnvDrop/
├── envdrop_videos_00.tar        # <video_id>.mp4 clips
│   ...                          # (15 shards, ~270 GB total)
├── envdrop_videos_14.tar
├── envdrop_motion.json          # primary training annotation (146,304 records)
└── annotations.json             # video_id -> instruction index / preview

Usage

python
from huggingface_hub import snapshot_download

# annotations only (small)
snapshot_download("Rithvik762/VLNCE-EnvDrop", repo_type="dataset",
                  allow_patterns=["*.json"])

# full dataset incl. video tars (~270 GB)
snapshot_download("Rithvik762/VLNCE-EnvDrop", repo_type="dataset")

After download, extract the shards (e.g. for f in envdrop_videos_*.tar; do tar xf "$f"; done). Videos and envdrop_motion.json must be kept together — the JSON references video_ids that live inside the tarballs.

License

Released under the MIT license.