rubatotree/pick-black-cat-act-miracle_256

Pick black cat into green SIGGRAPH mug — ACT

This deployment checkpoint accepts SO101 arm positions in degrees and gripper positions in 0–100. It returns absolute position targets in the same units. Set --robot.use_degrees=true and load BOTH saved processor pipelines. The model weights are unchanged from the completed 50000-update training run; the input/output normalization statistics have been converted together.

Use the existing lerobot-rollout command with --policy.path=rubatotree/pick-black-cat-act-miracle_256 and explicitly add --robot.use_degrees=true --fps=25. Cameras remain front and side, RGB 640×480 at 25 Hz. Set the control-loop FPS as well as the camera FPS: rollout otherwise defaults to 30 Hz. Observation state and action order: shoulderpan, shoulderlift, elbowflex, wristflex, wrist_roll, gripper. The action chunk has 100 targets; replan after 5 actions with current observations. These are absolute positions, not deltas. No extra radian-to-degree conversion should be added around this adapted checkpoint.

Gripper alignment

The mapping uses the supplied hardware encoder range 2016–3578 (1562 ticks), with p = 41.724679382926 * q + 7.282333466740, where q is the trained URDF gripper angle in radians and p is the LeRobot gripper value. The pinned model's nominal closed endpoint (-0.174533 rad) is aligned with hardware 0. This endpoint alignment is an explicit modeling assumption, not a measurement of the real jaw zero. The hardware driver bounds gripper commands to [0,100]. This mapping is specific to the supplied gripper travel; a different robot calibration needs another mapping.

The official SO101 model documentation states that the LeRobot gripper 0–100 mapping is not yet represented in its URDF/MuJoCo files. The arm uses the nominal new-calibration mid-range zero convention. See deployment_units.json for assumptions and hashes.

Validation and training

The original training and offline_evaluation.json use radians throughout. Dataset: rubatotree/miracle-pick-black-cat-256 at 85b50d49b5a024fc5f959e824f97917fd3a7bcc9. Training records and data are unchanged. deployment_validation.json separately checks the adapted processors, strict checkpoint reload, identical model hashes and 6 real network frame probes. Maximum recovered action difference was 4.47e-07 rad; this tests software equivalence, not physical task success.

The original radian-interface miracle_256 policies moved to a central pose and stopped in the user's hardware tests without an interface adapter. This release fixes that confirmed unit mismatch. The user subsequently reported severe jitter and failure to reach the grasp after the observation motion with the unit-adapted deployment. No successful real-robot trial has been established. Nominal jaw-zero residuals, visual domain mismatch and closed-loop behavior remain to be evaluated; no real-robot success rate is claimed. All synthetic derivatives share one source group.

The original radian-interface miracle_256 release is preserved on the radians-v1 branch. Use matching original processor files when replaying those weights in simulation. For this main deployment release, all model input/output calls use degree/percent units.

ACT continuity diagnostic and optional rollout settings

The saved default remains n_action_steps=5, temporal_ensemble_coeff=null. Two complete source-data episodes (600 observation frames) exposed jumps when switching five-step action chunks. Offline temporal ensembling with coefficient 0.01 reduced arm acceleration RMS by about 70–73%, while average tracking error remained similar. FP16 and FP32 results were similar. These comparisons replay recorded synthetic observations; they do not establish real-world success or diagnose camera delay.

To test the smoother execution on this ACT checkpoint, add both flags to the existing 25 Hz, degree/percent rollout command:

--policy.n_action_steps=1 \
--policy.temporal_ensemble_coeff=0.01

The two parameters must be set together. Official ACT configuration requires one action step for temporal ensembling. Predictions for the same execution time are combined across chunks; this is not averaging different time steps of the intended motion. Model weights, training data and unit processors remain unchanged.

Actual select_action with the saved processors and both overrides was checked separately (40 frame calls across FP32/FP16). This setting runs inference each control tick. On this server's RTX 4090, single-observation FP32 processing/inference took about 16–18 ms, excluding camera capture and motor I/O. Check actual loop timing on the deployment machine. control_continuity.json records the measured comparisons. Continued failure to grasp needs real observation/action timing and visual-domain diagnosis; smoothing alone is not a task-success fix.