Ahaskar04/viperx-3arm-handover-demo

COLA 3-Arm Sequential Handover — Scripted Demonstrations

1,500 simulated demonstrations of a sequential 3-arm handover task in MuJoCo, generated with a hand-tuned waypoint policy. Three ViperX 300s robot arms (A → B → C) cooperate to lift an object off a table, hand it between two arms, and place it into a box rigidly attached to the third arm.

This dataset is intended for multi-agent imitation learning, coordination research, and as the 3-arm extension of the 2-arm `cola-handover-demos`.

Task description

Three ViperX 300s arms arranged along the Y-axis on a shared table:

The chain proceeds through 6 environment phases:

PICKUP → HANDOVER_AB → PLACE_BC → COMPLETE → RETRACT_DONE → RETRACT_C_DONE

A success is a complete chain: A grasps and lifts → A hands to B mid-air → B rotates and drops the object into C's box → both arms retract.

Dataset stats

Failure mode notes. Most failures occur during the B→C placement phase (object misses the box on drop). The A→B handover stage itself succeeds in nearly all episodes that progress past pickup; pickup-phase timeouts are mostly cases where A grasped successfully but the env's strict is_arm_holding thresholds didn't fire to advance the env phase state. See Notes on success labels below.

File schema

Each episode is a single np.savez_compressed file with these arrays (T = timesteps in that episode):

Per-step observations

The three cameras are distinct global views (front / top / oblique side), not per-agent egocentric. The arm-to-camera mapping is by convention (A=front, B=top, C=side); all three views show all three arms.

Per-step world state

Per-episode metadata

Status is also encoded in the filename suffix: episode_{NNNN}_{success|dropped|timeout}.npz.

Videos

Alongside each .npz is a matching .mp4 showing an HD (720×1280) oblique side-view render of the same episode. Useful for visual debugging and dataset previews.

Loading

import numpy as np

ep = np.load("episode_0042_success.npz", allow_pickle=True)

T = int(ep["total_steps"])
images = {
    "front": ep["image_a"],   # (T, 256, 256, 3) uint8
    "top":   ep["image_b"],
    "side":  ep["image_c"],
}
proprio = {
    "a": ep["state_a"],   # (T, 10)
    "b": ep["state_b"],
    "c": ep["state_c"],
}
actions = {
    "a": ep["action_a"],  # (T, 7), in [-1, 1]
    "b": ep["action_b"],
    "c": ep["action_c"],
}
print(f"episode {ep['seed']}: {ep['status']}, {T} steps")

Streaming via the datasets library:

from huggingface_hub import snapshot_download

local_dir = snapshot_download(
    repo_id="Ahaskar04/viperx-3arm-handover-demo",
    repo_type="dataset",
    allow_patterns=["episode_*_success.npz"],   # e.g. only successes
)

Notes on success labels

The success flag is conservative. The env requires the full chain to complete before declaring success: A holds AND lifts → B holds → box contains object → both arms retract. Observed dataset characteristics:

• —A's pickup actually succeeds in nearly all "PICKUP timeout" episodes. The 261 timeouts marked with final_phase=PICKUP are mostly cases where A grasped and lifted the object, but the env's strict is_arm_holding predicate (gripper closed AND TCP within 5 cm of object) didn't latch firmly enough to advance the phase machine. The policy then proceeds through its own internal phases out of sync with the env, eventually opens A's gripper for the handover, and the object falls.
• —The B→C placement is the dominant real-failure mode (694 drops). Most of these are the object missing the box on drop, or B knocking the box on retraction.
• —The 545 _success.npz episodes are unambiguous: env confirmed full chain + retraction.

If you only care about "did A actually grasp and lift," you'll want to re-label with a more generous threshold using object_positions[:,2].max() and the gripper qpos in state_a[:,6].

Environment & data generation

Generated using the MuJoCo environment and waypoint policy in `cola-research/3arm-handover/` (private repo). Key components:

• —three_arm_handover_scene.xml: MuJoCo scene (3 ViperX arms, table, object, box rigidly attached to C's gripper, 4 global cameras + per-arm wrist cameras)
• —three_arm_handover_env.py: gym-style env with phase state machine, staged dense rewards, observation dict
• —scripted_policy.py: 14-phase finite-state-machine policy with reactive Cartesian tracking during handover and placement
• —run_scripted_policy.py: rollout driver

The collection ran on NSCC ASPIRE 2A (1× GPU, ~14 h walltime, 50 Hz control with mj_step substepping at the model timestep).

Object spawn distribution

Objects are spawned in a small box near A:

• —X: uniform in [-0.08, +0.08] m
• —Y: uniform in [-0.47, -0.43] m (close to A's reach)
• —Z: 0.40 m (constant, on table)

Quaternion is fixed identity (no spawn rotation). Both initial_object_pos (spawn) and object_positions[0] (after a 50-step settle) are recorded; they differ by the small drop during settling.

License

Released under MIT. The MuJoCo physics models for ViperX 300s come from the Trossen Robotics MuJoCo Menagerie (BSD-3).

Citation

If you use this dataset, please cite:

@misc{cola_3arm_handover_2026,
  title  = {COLA 3-Arm Sequential Handover Demonstrations},
  author = {Ahaskar Kashyap},
  year   = {2026},
  url    = {https://huggingface.co/datasets/Ahaskar04/viperx-3arm-handover-demo},
}

Related datasets

• —`demo4563u2i/cola-handover-demos` — the 2-arm precursor (1,500 episodes, 2 cameras)