angkul07/piper-retargeted

piper-retargeted

Human demonstrations retargeted to an AgileX Piper arm with the Fidelity Dynamics dt-pipeline (stage 6). Two datasets, one repo.

Each archive is self-contained: the observation video sits in the same clip directory as the trajectory it belongs to, so there is nothing to join.

Layout

<dataset>/clips/<clip_id>/observation.mp4   # RGB observation for this clip
<dataset>/clips/<clip_id>/retargeted.hdf5   # trajectories + per-frame IK error
<dataset>/clips/<clip_id>/report.json       # per-clip M1–M6 record, QA grades
<dataset>/manifest.json                     # aggregate stats + one row per clip
<dataset>/README.md

Video and trajectory are 1:1 per clip and share the clip's frame count and 30 fps timebase. Archives are uncompressed .tar — the MP4s are already compressed, so gzip would cost time and save nothing.

retargeted.hdf5:

Attributes carry joint_names, active_arms, parked_arms, arm_mode, n_arms, fps and the M4/QA records.

Robot

agilex_piper_bimanual. Stage 6 loads one single-arm MJCF and drives it once per arm with its own base placement, so "bimanual" needs no two-arm model. Joints joint1..joint6 plus jaw joint7 (joint8 is coupled = -joint7). The TCP is the grasp_frame fingertip midpoint, not Link6.

Workspace placement used WORKSPACE_CENTER_MODE=capmap: the centre is the reachability-index-weighted centroid of this arm's own pose-reachable voxels ([0.034, 0.000, 0.208] over 18,479 voxels), not a reference robot's vector rescaled. Scale 0.8811 (reach ratio 1.037; Piper 0.897 m vs reference 0.865 m).

Quality

Filter, don't discard. qa_verdict gates on the worst single arm joint's fraction of frames pinned at a limit, so one saturated DOF fails a clip whose tracking is otherwise good. manifest.json carries per-clip ik_mean_cm and saturation if you would rather threshold yourself. The stera set is substantially the weaker of the two.

Per-dataset notes

ego — both arms are active on every clip, but not equally busy: the right arm travels a median 1.00 m per clip against the left's 0.025 m and carries the error (4.25 vs 0.44 cm). A near-static arm can still saturate by resting against a limit; 139 of the 300 FAILs are left-arm.

stera — single-arm (--arm right); the left arm is a constant park pose carrying no signal, so drop columns 7–13 for single-arm training. 29 of the 255 selected clips are absent because the human demonstrator used their left hand, dropping the right below the 50% tracking threshold; they are listed in manifest.json under skipped_clips. Because M3.5 re-solves the moving arm to avoid the parked one, 18 clips FAIL on selfcol for a collision that will not exist once the left arm is dropped.

Known limit

About 73% of Piper's joint saturation is wrist-driven (joint4 and joint5; joint5's range is only ±1.22 rad, the tightest on the arm). A workspace re-placement cannot fix an orientation the wrist cannot reach, which is why the teleop-supervised refit is disabled for these runs — measured, it accepted 2 of 150 clips. The remaining lever is orientation-side, not placement.