Modelpublic

RyanL22/pi05-rby1-wujihand2-synth-boxfull-30k

sourceHugging Faceapache-2.0updated 4d agoView on Hugging Face

0likes27downloads

Model Card

pi0.5 — RB-Y1 + Wujihand2, synthetic only (human-video, box_full idm-pass), step 30k (final)

LeRobot-native pi05 (v0.6.1) fine-tuned on only synthetic RB-Y1 videos generated from human demonstrations (RoboTryOn-human-2, `lerobot_box_full_idm_pass`, 384 ep, 20 fps), no real teleop data: 11 cells, shares ∝ sqrt(frames).

Checkpoint: step 30,000 (final) of a 30,000-step run (NAVER MLXP, 2x H200, batch 32 x 2 = 64).

54-dim state / action

Same layout as RyanL22/pi05-rby1-wujihand2-teleopv1-baseline-20k: joint_position[10:64] = rightarm0..6 (7) | right hand 20 | leftarm0..6 (7) | left hand 20. Wheels / torso / head dropped (constant); hold them at the recording pose at rollout. max_state_dim = max_action_dim = 54, tokenizer max_length = 320 (already in this repo's config).

Training settings

	value
vision encoder (SigLIP, 412.4M)	fine-tuned (not frozen)
image augmentation	photometric + affine, one draw replayed across the stereo pair
mirror augmentation	off
optimizer	AdamW, peak lr 2.5e-5, cosine decay to 2.5e-6, warmup 1000
precision	bfloat16, gradient checkpointing
chunk	50 actions @ 20 fps (2.5 s), `n_obs_steps=1`
normalization stats	quantiles; constant dims widened (gate: q99 - q01 >= 1e-3)

Inputs

—observation.images.base_0_rgb <- left ZED view, 512x288; observation.images.left_wrist_0_rgb <- right ZED view, 512x288
—observation.state — 54 dims; action — 54 dims, absolute joint targets

Tasks (11): move the box to the right, starting with the left hand and finishing with the right hand / move the cup aside, then pick up the ball from table and put it in the white box / open the air fryer with the left hand, pick up the apple from the table with the right hand and put it in the air fryer, then close the air fryer with the left hand / pick up the ball from table and put it in the white box / pick up the bottle from table and put it in the white box / pick up the bottle from table and put it on the shelf at front / pick up the brush with the right hand and wipe away the black dot / pick up the coffee pot from table and put it in the white box / pick up the doll from table and put it in the white box / pick up the small box from table and put it in the white box / pour the coffee from the bottle into the red cup

Initial state (frame 0) — set the robot here before a rollout

All synthetic episodes start from one pose (per-task arm means within 0.26 rad of the common mean, std ≤ 0.19 rad), so use this synthetic common start for every task. Values are rad in the 66-dim robot order; the 54 policy dims are joint_position[10:64]. Wheels/torso/head are the recording constants and are not fed to the policy.

group	values
wheels (0:4)	0 0 0 0
torso (4:10)	+0.000 +0.524 -1.047 +0.524 +0.000 -0.000
right_arm (10:17)	+0.407 -0.158 +0.267 -1.878 -0.496 -0.149 -1.304
right hand (17:37)	+0.356 -0.132 +0.004 +0.059 +0.180 +0.060 +0.056 +0.162 +0.134 +0.074 +0.059 +0.157 +0.086 +0.055 +0.063 +0.169 +0.045 +0.086 +0.071 +0.188
left_arm (37:44)	+0.479 +0.044 -0.370 -1.891 +0.312 -0.299 +1.245
left hand (44:64)	+0.210 +0.027 +0.012 +0.059 +0.116 +0.073 +0.044 +0.101 +0.128 +0.077 +0.047 +0.111 +0.064 +0.056 +0.030 +0.110 +0.007 +0.074 +0.035 +0.091
head (64:66)	+0.000 +0.870

Real teleop episodes start from a more extended pose: rightarm `+0.522 -0.020 +0.267 -2.078 -0.005 +0.006 -1.592`, leftarm +0.528 +0.015 -0.268 -2.067 +0.008 +0.001 +1.591, hands ≈ open. Both starts are in-distribution for the 4 real tasks; for the synthetic-only tasks use the synthetic start.

Load

python

from lerobot.policies.pi05.modeling_pi05 import PI05Policy
policy = PI05Policy.from_pretrained("RyanL22/pi05-rby1-wujihand2-synth-boxfull-30k")