Datasetpublic

nevertmr/so101_coffee_rollouts

so101_coffee_rollouts 184 on-policy rollouts of a fine-tuned vision-language-action policy on an SO-101 arm, running a coffee-making routine as 10 separately instructed steps — with a success / fail / unstable label on every episode. Unlike teleoperated demonstration sets, every trajectory here was produced by the policy itself, so the failures are the policy's own. Recorded over 15 consecutive runs of the routine, with retries kept in place: when a step failed, the operator… See the full description on the dataset page: https://huggingface.co/datasets/nevertmr/so101_coffee_rollouts.

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

0likes69downloads

Dataset Card

so101coffeerollouts

184 on-policy rollouts of a fine-tuned vision-language-action policy on an SO-101 arm, running a coffee-making routine as 10 separately instructed steps — with a success / fail / unstable label on every episode.

Unlike teleoperated demonstration sets, every trajectory here was produced by the policy itself, so the failures are the policy's own. Recorded over 15 consecutive runs of the routine, with retries kept in place: when a step failed, the operator relabelled it and the arm attempted the same step again, so failures and their subsequent recoveries sit next to each other in the episode order.

This dataset was created using LeRobot.

Overview


Episodes / frames	184 / 38,718 (21.5 min)
Outcomes	149 success · 31 fail · 4 unstable
Composition	15 runs × 10 instructed steps, retries included
Policy	`aailabkaist/pi05_coffee_NorRec_RW_10k` (π0.5 fine-tune), async execution, chunksizethreshold 0.7
Robot	so101follower, 6-DoF — `shoulderpan`,` shoulderlift`, `elbowflex`,` wristflex`, `wristroll`,` gripper`
Cameras	`observation.images.front`, `observation.images.wrist` — 640×480, h264
fps	30
Format	LeRobotDataset v3.0

Episode durations are not meaningful as a quality signal: the operator ended each episode by hand once the step had clearly succeeded or failed.

Outcome labels

episode_labels.csv carries one row per episode, aligned to episode_index:

column	meaning
`episode_index`	index in this dataset (0–183)
`run`	source run, `coffee_new01` … `coffee_new15`
`source_episode`	episode id inside that run
`cycle`	routine cycle within the run
`step_index`	which of the 10 steps (1–10)
`instruction`	natural-language instruction given to the policy
`outcome`	`success`, `fail`, or `unstable`
`memo`	operator note, when one was written (Korean)

unstable means the step reached its goal but only after visibly erratic behaviour — for example pressing, retreating, then returning to press again.

Steps

Each run executes the routine below in order. A failed step is retried immediately, so a run contains 11–15 episodes rather than exactly 10.

step	instruction	episodes	success	fail	unstable	first-attempt success
1	pick up the cup on the blue circle	16	15	1	0	14/15
2	place the cup to the left coffee machine	15	15	0	0	15/15
3	press the blue button on the left coffee machine	19	13	4	2	10/15
4	pick up the cup on the blue circle	18	15	3	0	13/15
5	place the cup to the right coffee machine	16	15	1	0	14/15
6	press the blue button on the right coffee machine	33	14	18	1	3/15
7	pick up the cup on the left coffee machine	18	17	1	0	14/15
8	place the cup on the left pink circle	17	15	2	0	14/15
9	pick up the cup on the right coffee machine	16	16	0	0	15/15
10	place the cup on the left pink circle	16	14	1	1	13/15

Failures concentrate on step 6: it accounts for 18 of the 31 failures, and succeeded on the first attempt in only 3 of 15 runs. Steps 1–10 are the portion of the routine that was actually rolled out; the two closing steps of the full 12-step task are not present.

Episode ranges

Episodes are ordered run by run, and within a run in the order they were recorded.

run	episode range	episodes	success	fail	unstable
coffee_new01	0–13	14	10	4	0
coffee_new02	14–24	11	10	1	0
coffee_new03	25–36	12	10	2	0
coffee_new04	37–49	13	10	3	0
coffee_new05	50–64	15	11	3	1
coffee_new06	65–77	13	10	3	0
coffee_new07	78–88	11	10	1	0
coffee_new08	89–101	13	10	3	0
coffee_new09	102–112	11	10	1	0
coffee_new10	113–124	12	9	2	1
coffee_new11	125–135	11	9	1	1
coffee_new12	136–149	14	11	3	0
coffee_new13	150–161	12	10	2	0
coffee_new14	162–172	11	10	1	0
coffee_new15	173–183	11	9	1	1

16 further episodes were discarded at recording time (aborted or mis-triggered) and are not included.

Repository contents

path	what it holds
`data/`, `videos/`, `meta/`	the merged LeRobot v3.0 dataset described above
`episode_labels.csv`	per-episode outcome labels
`runs/coffee_newNN/`	the same episodes kept split by run, plus each run's raw event log (`run_meta.json`)
`raw/coffee_newNN_raw.tar`	per-frame JPEGs and `steps.jsonl` (state, action, timing) as recorded, including the 16 discarded episodes

Uses

—Failure detection and success classification from video, with per-episode ground truth
—Preference or filtered-imitation learning: failure → retry-success pairs within a run share the same scene and instruction
—Studying where a fine-tuned VLA breaks down on a long-horizon routine

Companion repository

`nevertmr/so101_coffee_subtask` holds 431 human teleoperated demonstrations of the same routine — useful as the demonstration counterpart to these on-policy rollouts.