Aether258/pi05_bi_bread_all_step6000

pi05_bi — bread (01+02 merged), step 6000

openpi pi05_bi checkpoint for a bimanual bread-and-ketchup task with tactile inputs. Step 6000 (1.94 epochs), held-out validation loss 0.0389.

The lowest held-out loss through step 10000 is step 10000 at 0.0384 (Aether258/pi05_bi_bread_all_step10000), though the two are within noise of each other.

Task

Single instruction for every episode (both source repos already agreed on this string, so no unification was needed at merge time):

Firstly, use the right hand to pick up the bread and put it on the table. Secondly, use the left hand to pick up the ketchup and squeeze it onto the bread.

Data

LeRobot v2.1, 30 fps, robot_type=bimanual, images embedded in the parquet files (total_videos=0). Six camera streams: camera0, camera1, and four tactile sensors (tactile_left_0/1, tactile_right_0/1). Mean episode length ~568 frames -- notably shorter than the sibling two_tubes (~788) and task2 (~966) datasets, so an epoch here is only 3,097 steps.

Split

Episodes are held out per source repo (10%, seed 42) so the held-out set keeps the same source mix as train:

Normalization statistics (quantile q01/q99) are computed over the train split only.

Training

Validation curve

Flow-matching loss, 20 batches per split, evaluated on the same leading batches each time so successive points are comparable.

val_unseen fell steeply through step 6000 and has been flat since. The per-2000-step improvement was 0.0044, then 0.0016, then 0.0002, then 0.0003 -- the last two are within noise, since each validation pass covers only ~2,560 frames, which at ~568 frames per episode is about 4.5 episodes per split. In practical terms steps 6000, 8000 and 10000 (0.0389 / 0.0387 / 0.0384) are interchangeable.

The quantity that is actually moving is the gap, which narrowed monotonically across all five points: -0.0060, -0.0054, -0.0044, -0.0033, -0.0012. Over the last 2,000 steps val_seen improved 5.5% (0.0419 -> 0.0396) while val_unseen moved 0.0003. That is the signature of capacity going into fitting the training episodes rather than transferring -- overfitting has begun, it simply started from a negative gap and has not yet crossed zero.

Two things distinguish this run from the sibling two_tubes_0102 and task2_all runs, both of which had clearly turned by 1.2-1.9 epochs:

1. 1.It kept improving past 1.9 epochs. At step 6000 (1.94 epochs) val_unseen was still dropping steeply, where the other two had already plateaued. The turn here came at roughly 2.0-2.6 epochs instead.
2. 2.The gap is negative throughout (val_unseen below val_seen). This is not evidence of unusually good generalization -- with only ~4.5 episodes sampled per split, the sign of the gap is dominated by which particular episodes landed where. Only the val_unseen absolute trend, and the direction the gap moves, carry information.

Note the training loss is measured on augmented images (random crop to 95%, +-5 deg rotation, colour jitter, applied to all six streams including the four tactile ones) while validation runs with train=False, which skips augmentation. The two columns are therefore not directly comparable.

Loss values are not comparable across datasets -- normalization statistics are computed per dataset, so this run's 0.0387 does not mean it is "better" than two_tubes_0102's 0.0537.

Contents

checkpoint/
  params/                      # inference weights
  train_state/                 # optimizer state, for resuming
  assets/bread_all/
    norm_stats.json            # computed over the train split only