endoard/grab_cup_2cam_v6

grabcup2cam_v6 — SO-101, two cameras

LeRobot v3.0 dataset for the SO-101 arm. Task prompt (exact string): `Grab the white cup`. Cameras fixed and handeye, 30 fps.

grab_cup_2cam_v4 with the dead episode, the frozen-action lead-in, and the long standing-still stretches removed.

Why

A policy trained on v4 drives the arm to a correct pose and then freezes there. Half of v4 is the arm standing still; a policy that sees one observation at a time has no clock, so "pause here for two seconds" and "stay here forever" are the same input, and the majority vote says stay.

The failure unit is a dead chunk: pi0.5 predicts 16 actions at action_stride=2, a window of 32 source frames. If the arm never moves in those 32 frames, the sample teaches "stay" and says nothing about when to resume. v6 has none.

What is unusual about this dataset

The `timestamp` column has holes, on purpose. Frames were dropped from the middle of episodes and the remaining timestamps were left at their original values. This is what keeps the videos correct without re-encoding: LeRobot decodes a row's image at from_timestamp + row.timestamp, addressed by the value of the timestamp, never by the row's position.

So: *do not renumber `timestamp`, and do not touch `videos//fromtimestamp`.** Doing either pairs every state with the wrong image, silently. `timestamp == frameindex / fps` holds for every row and is the invariant to preserve.

Action chunks are addressed by row offset instead, so a chunk can span a hole. That is safe here because only static stretches were removed: the largest drift across any dropped pause is 1.58°, against 1.85° for an ordinary chunk step (p75). The discontinuity is smaller than normal motion. The demonstrated wait disappears; the trajectory does not jump.

Episode count is unchanged at 145. That matters: splitting episodes instead would have pushed the chunks that run off an episode end — filled with a repeated last action, and trained on as real, since nothing in openpi reads action_is_pad — from 2.4% to 9.5% of the loss.

Training with openpi

Norm stats are included under openpi_assets/; you do not need to recompute them. Do not reuse v4's — the distributions shifted.

# 1. make the dataset resolvable: openpi calls LeRobotDataset(repo_id) with no root
mkdir -p "$HF_LEROBOT_HOME/so101"
ln -s /path/to/grab_cup_2cam_v6 "$HF_LEROBOT_HOME/so101/grab_cup_2cam_v6"

# 2. drop in the norm stats
cp -r openpi_assets/so101 openpi/assets/pi05_so101_v6/

# 3. train  (pi0.5, action_dim=32, action_horizon=16, 30k steps, batch 32)
uv run python scripts/train.py pi05_so101_v6 --exp-name <name>

# 4. serve
uv run scripts/serve_policy.py policy:checkpoint \
    --policy.config=pi05_so101_v6 --policy.dir=/path/to/checkpoint

The openpi config keeps repo_id="so101/grab_cup_2cam_v6", a local name rather than this Hub id: asset_id is derived from it and decides where the norm stats are looked up. --policy.config must match the config used for training, or serve_policy fails with FileNotFoundError on norm_stats.json.

To regenerate the stats instead: uv run python scripts/compute_norm_stats.py --config-name pi05_so101_v6

Regenerating v6

python clean_dataset.py --src grab_cup_2cam_v4 --dst grab_cup_2cam_v4_clean --write
python warp_dataset.py  --src grab_cup_2cam_v4_clean --dst grab_cup_2cam_v6  --write

warp_dataset.py keeps at most 29 frames of each pause. The limit is action_horizon × action_stride − 3: after a hole the first surviving row is still static relative to its predecessor, so motion only reappears one row later, and a chunk starting on a pause's first frame must still reach it.

Supersedes

grab_cup_2cam_v5 — same goal, but cut by splitting episodes into 498 shorter ones. Use v6.