Datasetpublic

ASethi04/droid-trajectories-5k

DROID Visual Trajectories (5K Episodes) — LeRobot Format Visual object trajectories extracted from 4506 episodes of the DROID dataset using Gemini Robotics-ER + SAM2. How to load images/videos Images are not embedded in this dataset. Each row contains reference paths to the source videos in cadene/droid_1.0.1: from datasets import load_dataset from huggingface_hub import hf_hub_download ds = load_dataset("ASethi04/droid-trajectories-5k", split="train") row =… See the full description on the dataset page: https://huggingface.co/datasets/ASethi04/droid-trajectories-5k.

sourceHugging Faceapache-2.0updated 6mo agoView on Hugging Face

0likes454downloads

Dataset Card

DROID Visual Trajectories (5K Episodes) — LeRobot Format

Visual object trajectories extracted from 4506 episodes of the DROID dataset using Gemini Robotics-ER + SAM2.

How to load images/videos

Images are not embedded in this dataset. Each row contains reference paths to the source videos in cadene/droid_1.0.1:

python

from datasets import load_dataset
from huggingface_hub import hf_hub_download

ds = load_dataset("ASethi04/droid-trajectories-5k", split="train")
row = ds[0]

# Get the source video
video_path = hf_hub_download(
    "cadene/droid_1.0.1",
    row["droid_video_exterior_1_left"],  # e.g. "videos/chunk-000/observation.images.exterior_1_left/episode_000008.mp4"
    repo_type="dataset",
)

# Or load the full source parquet (with all DROID columns)
droid_ep_id = row["droid_episode_id"]  # e.g. 8
chunk = droid_ep_id // 1000
source_parquet = hf_hub_download(
    "cadene/droid_1.0.1",
    f"data/chunk-{chunk:03d}/episode_{droid_ep_id:06d}.parquet",
    repo_type="dataset",
)

Stats

—Episodes: 4506
—Total Frames: 943315
—Unique Tasks: 1429
—Cameras: exterior1left, exterior2left, wrist_left (referenced)
—FPS: 15

Schema

Trajectory columns (from Gemini ER + SAM2 pipeline)

Column	Type	Description
`trajectory_xyxy`	float32[4]	Tracked object bbox [x1, y1, x2, y2] in pixels
`trajectory_center_x`	float32	Object mask centroid X (pixels)
`trajectory_center_y`	float32	Object mask centroid Y (pixels)
`trajectory_labels`	list[string]	Object labels being tracked

Robot state (from cadene/droid_1.0.1)

Column	Type	Description
`joint_position`	float32[7]	7-DOF joint angles
`gripper_position`	float32	Gripper state
`actions`	float32[8]	Action commands (7 joints + gripper)

References to source data

Column	Type	Description
`droid_episode_id`	int64	Original episode ID in cadene/droid_1.0.1
`droid_video_exterior_1_left`	string	Path to exterior1left video in cadene/droid_1.0.1
`droid_video_exterior_2_left`	string	Path to exterior2left video in cadene/droid_1.0.1
`droid_video_wrist_left`	string	Path to wristleft video in cadene/droid1.0.1

Metadata

Column	Type	Description
`language_instruction`	string	Task description
`timestamp`	float32	Frame timestamp (seconds)
`frame_index`	int64	Frame number within episode
`episode_index`	int64	Sequential episode number (0-indexed)
`index`	int64	Global row index
`task_index`	int64	Task ID (see meta/tasks.jsonl)

Trajectory Pipeline

1.Object Extraction: Parse task instruction to identify primary + related objects
2.Detection: Gemini Robotics-ER (gemini-robotics-er-1.5-preview) with task-aware bounding box prompts
3.Tracking: SAM2 video predictor with periodic re-detection every 60 frames
4.Trajectory: Per-frame mask centroid and bounding box

Source

—Robot data & videos: cadene/droid_1.0.1
—Format inspired by: brandonyang/droid_120