jren313/g1-pipette-tip-teleop-phase

g1-pipette-tip-teleop-phase

Unitree G1 + Inspire RH56 teleoperation: attach a red pipette tip from a rack to a pipette, then eject it into a disposal box. LeRobot v2.1, 73 episodes / 68,585 frames @ 60 fps.

Two things distinguish this from the earlier releases: a detector-tracked zoom stream on the tip rack, and per-frame phase sentences as a second language channel.

Streams

The red tip is 2-4 px in the ego frame and survives the usual 224x224 model resize as ~11 pixels; in the crop it survives as ~45. That ratio is the whole reason the second view exists.

Phase labels

annotation.human.phase is a per-frame column pointing at rows 3-5 of tasks.jsonl:

annotation.human.task_description / .desc / .steps (rows 0-2) are unchanged, so single-sentence recipes train from this dataset untouched.

Three phases, not four

Earlier revisions split P3 into a lift and a carry-and-eject. They were merged on 2026-08-18. The lift ran a median 0.95 s -- 6.4% of frames, and its boundary was the only one set by an arbitrary knob rather than an extremum: bottom is an argmin and eject an argmax, but "lifted" meant z recovered through half its climb, and sweeping that fraction across a plausible range moved the cut a median 0.26 s, 27% of the phase it bounded. A label that uncertain over that few frames teaches little, and it cost a transition that a serving-time phase machine would have to detect.

Descending onto the tip is P2 and rising off it is P3 either way, so the perceptual aliasing these sentences exist to break is unaffected: those two put the arm at nearly the same joint configuration in front of nearly the same picture, and 68% of look-alike frame pairs have opposed next-chunk arm directions.

Boundaries

Two cuts, recovered from the right wrist by forward kinematics on the G1 URDF: arrive (wrist settles at its left extreme) and bottom (lowest z before the ejector fires -- the insertion). Every threshold is a fraction of that episode's own range; absolute cuts do not transfer, because left-most wrist y spans -0.13..-0.07 m and lowest z spans 0.042..0.059 m across episodes.

meta/phase_events.json carries per episode a boundaries block (the two cuts), an events block (arrive, bottom, plus lift and eject retained for provenance -- neither is a boundary), shares, and a phase_ok flag -- 73/73 pass.

Eight episodes use hand-read timestamps: 7, 8, 24, 31, 36, 40, 50 and 58. In 7/24/36/40 the automatic events could not separate the phases at all. In 8/31/50/58 they were plausible but wrong on review -- arrive fired 1.4-5.2 s late in three of them (it triggers on "y within 15% of its left extreme", which lands late whenever the arm drifts further left later on), and in 31 the bottom argmin found the disposal reach instead of the insertion, 6.7 s early.

Splits

meta/splits.json records which set each episode belongs to, carried over from the 2026-08-13 split (seed 1234):

The training set used by the current recipes is train + new = 62 episodes. Every good episode is published; the split is a metadata filter rather than a deletion.

Provenance -- two captures, four drops, one disagreement

Four episodes of the original 64 are NOT here. In source episodes 40, 44, 45 and 55 the crop tracker locked on the forearm and wrist cuff, so tip_crop shows a silver tube for the whole episode instead of the rack. A crop that never contains the rack is a wrong sample rather than a weak one, so they are dropped.

Episodes are therefore renumbered contiguously (LeRobot requires 0..N-1) and the original numbers no longer index this set. meta/source_provenance.jsonl maps every episode back to its capture and source index.

The two captures disagree about the ejector. action.hand_right[5] (thumb_rot) rests at 500 and the press drives it up across the 2026-08-13 capture; it rests at ~956 and the press drives it down across all 13 of the 2026-08-17 one. Training on the union makes that channel bimodal. Hold the captures apart, normalise the convention, or accept that the eject is the one action dim that cannot be learned cleanly from the mixture.

Several hand channels are constant by construction and carry no signal: action.hand_left (all), observation.state.hand_right, observation.force.*, observation.state.neck.

Load

from lerobot.common.datasets.lerobot_dataset import LeRobotDataset
ds = LeRobotDataset("jren313/g1-pipette-tip-teleop-phase")