Ahaskar04/aloha-handover-no-mocap

ALOHA 2-Arm Handover (no mocap markers)

163 successful scripted demonstrations of a bimanual box handover in MuJoCo, recorded on the ALOHA / ViperX 300 dual-arm platform. Arm A picks a box off the table and hands it to arm B, which retracts with it.

Why "no mocap"

This is a re-collection of `Ahaskar04/aloha-handover-data` with a visual confound removed.

The scripted expert drives the arms through MuJoCo mocap bodies (left/target, right/target) — 2 cm spheres that are moved to each phase's IK goal, with the arm following after. In the original dataset those spheres were rendered into every camera frame, occupying ~6% of the wrist view (≈3,800 of 65,536 pixels) and moving to exactly where the arm was about to go.

That gives a behaviour-cloning policy an easy shortcut: track the green sphere instead of reasoning about the box and the arm's own pose. The shortcut breaks at inference, because nothing sets mocap_pos during a rollout — the spheres sit frozen at their reset position, so the policy's strongest cue simply stops moving.

Here the marker geoms are set to rgba="0 1 0 0" (alpha 0). Verified: moving a mocap body changes 0 pixels in all three cameras, against ~36,000 before.

Contents

Each episode_XXXX.h5 holds one episode. T varies per episode (mean 155, range 148–163); 25,290 timesteps in total.

Episode attribute: success (bool) — all episodes here are successes.

Action space

Indices 0–5 are joint angles in radians: waist, shoulder, elbow, forearm_roll, wrist_angle, wrist_rotate. Index 6 is the gripper, which takes only two values: 0.002 (closed) or 0.037 (open) — the actuator's ctrlrange.

Treating the gripper as continuous is a trap. Its range spans 0.035 while the joints span 0.11–0.91 in standard deviation, so under a uniformly averaged L1 loss it contributes ~2% of the gradient and is effectively ignored. It is better handled as binary classification.

Phases

approach → open_gripper → descend → grasp → lift
         → move_arm_B → approach_b → grip_b → release_a → retract_b

descend and grasp are the shortest phases (~10 and ~2 steps) and carry the largest joint motion, making them the hardest transitions to imitate.

Recording details

• —Control rate 50 Hz — the simulator runs at 500 Hz (timestep = 0.002) and one frame is recorded every 10 physics steps. A policy replaying these actions must hold each one for 10 steps to match.
• —Box spawn — uniform in x ∈ [-0.12, 0.12], y ∈ [-0.10, 0.10], at z = 0.03. Evaluating outside this range is out of distribution.
• —Initial pose — the neutral_pose keyframe, i.e. mj_resetDataKeyframe, not mj_resetData (which leaves every joint at zero, ~1.16 rad away at the elbow).
• —Expert is an IK-based scripted policy (mink + DAQP).
• —163 successes out of 200 attempts; failures were discarded.

Loading

import h5py
from huggingface_hub import snapshot_download

path = snapshot_download("Ahaskar04/aloha-handover-no-mocap", repo_type="dataset")

with h5py.File(f"{path}/episode_0000.h5", "r") as f:
    images = f["image_wrist_a"][:]   # (T, 256, 256, 3) uint8
    actions = f["action_a"][:]       # (T, 7) float64

Known limitations

• —Expert trajectories only. Every state lies on a successful IK path, so there are no examples of recovering from a mistake. Policies trained by plain behaviour cloning drift off-distribution during closed-loop rollout and have no demonstration to fall back on.
• —Single task, single object, fixed camera placement.
• —163 episodes is small for this setting.