MLeggiero/g1-inspire-pipette-tip

G1 + Inspire — "eject the pipette tip into the rack" (GR00T N1.7 / G1_INSPIRE)

Teleoperated Unitree G1 (29-DoF) + Inspire RH56DFTP hands manipulation data, converted to the GR00T-flavored LeRobot v2.1 format for fine-tuning GR00T N1.7 as a custom NEW_EMBODIMENT (here called G1_INSPIRE).

Same schema as `MLeggiero/g1-gr00t-inspire-pick_and_place`, with two additions: `observation.effort` (per-joint torque) and native 1280×720 ego-view video and depth instead of 424×240. Read Known limitations before training on this — the hand proprioception channel is not usable.

Summary

• —Task (1): eject the pipette tip into the rack
• —Episodes / frames: 68 / 61,609 @ 60 fps; success flag per episode in meta/episodes.jsonl
• —Sessions: 3 capture sessions on 2026-08-03 (20260803_1445, 20260803_1455, 20260803_1511)

• —Camera: onboard Intel RealSense D435 ego-view (observation.images.ego_view, 1280×720, h264)
• —observation.state: 63-dim · action: 30-dim · observation.tactile: 34-dim · observation.effort: 29-dim

The task

The robot grips a micropipette in its right hand, moves it over a 96-well tip rack, and presses the tip ejector with its thumb to release the tip into the rack. The four fingers stay clamped on the pipette barrel throughout; the thumb is the only actuated hand DoF.

Data fields & vector index

observation.state (float32[63]) — proprioception (+ pooled tactile)

The baseline meta/modality.json uses only 0:29 (tactile-free); the tactile meta/modality.tactile.json uses 0:63.

action (float32[30]) — GR00T prediction target

Arm/hand action targets are stored as absolute joint angles (radians), matching the upstream g1-gr00t-inspire-pick_and_place dataset. Lower body: GR00T does not output leg joints — it emits a base velocity (navigate_command) + base_height, which the WBC turns into leg motion.

observation.effort (float32[29]) — per-joint torque

Estimated joint torque (dof_tau) in N·m, in the canonical 29-joint order (left_leg 0–5, right_leg 6–11, waist 12–14, left_arm 15–21, right_arm 22–28) — i.e. the full-body joint order, not the arm-first ordering used by state/action. Not referenced by either modality config; use it as an auxiliary supervision or contact-inference signal. All 29 dims are live.

observation.tactile (float32[34]) — pooled tactile, also embedded in state[29:63]

Per hand, 17 regions in this order (per-region max-pool of the raw taxels, then log1p): little_tip, little_nail, little_pad, ring_tip, ring_nail, ring_pad, middle_tip, middle_nail, middle_pad, index_tip, index_nail, index_pad, thumb_tip, thumb_nail, thumb_middle, thumb_pad, palm → dims 0–16 = left hand, 17–33 = right hand.

Other parquet columns

meta/

• —info.json — schema (codebaseversion v2.1, fps 60, robottype unitree_g1, per-feature shapes/names).
• —episodes.jsonl — per episode: episode_index, tasks, length, success (bool), source (original <date>_<time>/episode_NNNN session).
• —tasks.jsonl — the single task string.
• —stats.json — per-feature mean/std/min/max/q01/q99 (GR00T normalization), including observation.effort.
• —modality.json (baseline) / modality.tactile.json (tactile) — GR00T modality slices.

Sidecars (raw, not referenced by the modality configs)

• —tactile/episode_0000NN.npz → left, right: (T, 1062) uint16 raw taxels (0…~65535). The 1062 split into the 17 regions above with grid shapes: tip 3×3 (9), nail 12×8 (96), pad 10×8 (80) for little/ring/middle/index; thumb = tip 3×3 (9) + nail 12×8 (96) + middle 3×3 (9) + pad 12×8 (96); palm 8×14 (112). (Σ = 1062.)
• —depth/episode_0000NN.npz → depth: (T, 720, 1280) uint16, depth in millimeters, spatially registered to the ego-view frames.

Units & conventions

Arm/waist/hand angles in radians; hands are Inspire RH56DFTP DoFs mapped from raw 0–1000 registers to radians via the joint limits. base_height in meters, navigate_command in m/s & rad/s (robot-local frame). Effort in N·m. Tactile in observation.* is log1p of per-region max-pooled raw taxels; the tactile/ sidecar holds the un-pooled uint16 taxels.

Known limitations

These are properties of the capture, not of the conversion. Stated plainly so you can decide what to train on.

1. 1.Hand proprioception is dead. state[14:26] (all 12 hand dims, both hands) is constant for every frame of every episode. The recorder's hand-state stream was not updating during capture — its source timestamp is frozen and ~70 h stale. Do not use hand state as an input feature; mask it or use modality.json's 0:29 slice with the hand block excluded.
2. 2.Action hand channel is nearly constant. action[20:26] (right hand) varies in only 2 of 6 dims (thumb_pitch, thumb_yaw); the four fingers are pinned closed on the pipette barrel. action[14:20] (left hand) is fully constant — the left hand is unused and held open.
3. 3.Single-arm task. Only the right arm manipulates. Left-arm state/action are present and do vary slightly (postural), but carry no task signal.
4. 4.All episodes are labelled `success: true`. The capture contains no negative examples, so success carries no discriminative information here.
5. 5.One very short episode (~119 frames ≈ 2 s) is present and is likely an aborted take; filter on length in meta/episodes.jsonl if that matters.
6. 6.Body state, effort, tactile and vision are all healthy — state[0:14], state[26:29], all 29 effort dims, both tactile blocks and the ego-view video vary normally. The usable learning signal is arm + waist + tactile + effort + vision.

Fine-tuning (GR00T N1.7)

python gr00t/experiment/launch_finetune.py \
  --base-model-path nvidia/GR00T-N1.7-3B \
  --dataset-path <this dataset> \
  --embodiment-tag NEW_EMBODIMENT \
  --modality-config-path g1_inspire_modality_config.py \
  --num-gpus 1 --global-batch-size 4

Note that observation.images.ego_view here is 1280×720, not the 424×240 of g1-gr00t-inspire-pick_and_place. GR00T resizes internally, but the two datasets are not byte-compatible; co-training requires a resize step.

Provenance & license

Author: MLeggiero. Data collected on a Unitree G1 + Inspire RH56DFTP teleoperation rig, using the open-source collection stack from the YanjieZe repository (tooling credit only). Captured 2026-08-03. Released under the MIT License.