Datasetpublic

nevertmr/so101_coffee_subtask

so101_coffee_subtask 431 teleoperated SO-101 demonstrations of a coffee-making routine, recorded as 12 separately instructed steps, each captured with two cup variants — a red cup and a white one. Every step is a short, self-contained manipulation with its own natural-language instruction, so the set can be used either as 24 single-skill datasets or as one long-horizon routine. This dataset was created using LeRobot. Overview Episodes / frames… See the full description on the dataset page: https://huggingface.co/datasets/nevertmr/so101_coffee_subtask.

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

0likes103downloads

Dataset Card

so101coffeesubtask

431 teleoperated SO-101 demonstrations of a coffee-making routine, recorded as 12 separately instructed steps, each captured with two cup variants — a red cup and a white one.

Every step is a short, self-contained manipulation with its own natural-language instruction, so the set can be used either as 24 single-skill datasets or as one long-horizon routine.

This dataset was created using LeRobot.

Overview


Episodes / frames	431 / 99,472 (0.92 h)
Composition	12 steps × 2 cup colours = 24 contiguous blocks
Episode length	mean 231 frames (7.7 s), median 223, range 135–434
Distinct instructions	10
Robot	sofollower, 6-DoF — `shoulderpan`,` shoulderlift`, `elbowflex`,` wristflex`, `wristroll`,` gripper`
Cameras	`observation.images.front`, `observation.images.wrist` — 640×480, av1
fps	30
Format	LeRobotDataset v3.0

Trajectories

End-effector (pinch-point) paths of every episode, obtained by forward kinematics from the recorded joint states. Top panel is a top-down X–Y view with the table, the two coffee machines and the two circular markers drawn in; bottom panel is the side X–Z view. The robot base is at the origin (▲). Red lines are red-cup episodes, blue lines white-cup ones.

Three steps to a row — the full 12-step routine in order:

Steps 1–3 — pick up the cup on the blue circle → place the cup to the left coffee machine → press the blue button of the left coffee machine

steps 1-3

Steps 4–6 — pick up the cup on the blue circle → place the cup to the right coffee machine → press the blue button of the right coffee machine

steps 4-6

Steps 7–9 — pick up the cup on the left coffee machine → place the cup to the left pink circle → pick up the cup on the right coffee machine

steps 7-9

Steps 10–12 — place the cup to the left pink circle → pick up the cup near the blue circle → place the cup on the blue circle

steps 10-12

All four 3-step rows are in `viz/`.

Trajectory video

![12-step trajectory video, 4 views (4K)](https://youtu.be/ey8GLF_dZ2I)

▶ Watch on YouTube (4K): https://youtu.be/ey8GLF_dZ2I

Episode ranges

The cup colour is not part of the instruction string, so this table is what maps an episode to its step and cup. Every combination is one contiguous block of episode_index:

`episode_index`	Step	Cup	Episodes	Frames	Instruction
0–19	1	red	20	6,058	`pick up the cup on the blue circle`
20–39	1	white	20	5,204	`pick up the cup on the blue circle`
40–59	2	red	20	5,652	`place the cup to the left coffee machine`
60–79	2	white	20	4,883	`place the cup to the left coffee machine`
80–99	3	red	20	5,231	`press the blue button of the left coffee machine`
100–119	3	white	20	4,838	`press the blue button of the left coffee machine`
120–139	4	red	20	4,323	`pick up the cup on the blue circle`
140–159	4	white	20	4,093	`pick up the cup on the blue circle`
160–175	5	red	16	2,886	`place the cup to the right coffee machine`
176–192	5	white	17	3,461	`place the cup to the right coffee machine`
193–209	6	red	17	3,859	`press the blue button of the right coffee machine`
210–226	6	white	17	3,989	`press the blue button of the right coffee machine`
227–243	7	red	17	4,020	`pick up the cup on the left coffee machine`
244–260	7	white	17	3,638	`pick up the cup on the left coffee machine`
261–277	8	red	17	4,171	`place the cup to the left pink circle`
278–294	8	white	17	4,133	`place the cup to the left pink circle`
295–311	9	red	17	4,022	`pick up the cup on the right coffee machine`
312–328	9	white	17	3,490	`pick up the cup on the right coffee machine`
329–345	10	red	17	3,920	`place the cup to the left pink circle`
346–363	10	white	18	3,999	`place the cup to the left pink circle`
364–380	11	red	17	3,488	`pick up the cup near the blue circle`
381–397	11	white	17	3,576	`pick up the cup near the blue circle`
398–413	12	red	16	3,058	`place the cup on the blue circle`
414–430	12	white	17	3,480	`place the cup on the blue circle`
			431	99,472

python

from lerobot.datasets.lerobot_dataset import LeRobotDataset

ds       = LeRobotDataset("nevertmr/so101_coffee_subtask")
step3_red = LeRobotDataset("nevertmr/so101_coffee_subtask", episodes=list(range(80, 100)))

Steps and instructions

Steps 1 and 4 / 8 and 10 reuse an instruction: the routine makes two cups, so two of the instructions occur at two different points in the sequence. task_index is the index of the instruction, not of the step — there are 12 steps but 10 distinct instructions.

Step	`task_index`	Instruction	Episodes	Frames
1	0	`pick up the cup on the blue circle`	40	11,262
2	1	`place the cup to the left coffee machine`	40	10,535
3	2	`press the blue button of the left coffee machine`	40	10,069
4	0	`pick up the cup on the blue circle` (same as step 1)	40	8,416
5	3	`place the cup to the right coffee machine`	33	6,347
6	4	`press the blue button of the right coffee machine`	34	7,848
7	5	`pick up the cup on the left coffee machine`	34	7,658
8	6	`place the cup to the left pink circle`	34	8,304
9	7	`pick up the cup on the right coffee machine`	34	7,512
10	6	`place the cup to the left pink circle` (same as step 8)	35	7,919
11	8	`pick up the cup near the blue circle`	34	7,064
12	9	`place the cup on the blue circle`	33	6,538