AmeliaHE/wowrobo-so-arm101-bimanual-sim

SO-ARM101 Bimanual Lab

This public lab compares real dual-WOWROBO SO-ARM101 demonstrations with a contact-only MuJoCo mapping. It is a diagnostic tool, not a prerecorded success animation and not a controller for a physical robot.

Hardware-free Bimanual Twin Studio

/virtual is now a research-backed experiment surface rather than a single replay page. It never enumerates a serial port or contacts the local calibration bridge. Official SO-101 meshes are rendered locally with WebGL; rigid-body state comes from cloud MuJoCo. The page separates three runtimes visibly:

• —Browser Twin: low-latency rehearsal, joint mapping, visual randomization, material-sensitivity inspection, and local episode-contract export.
• —MuJoCo Rigid Lab: deterministic can/cube reset, step, observation, reward, and success checks through /api/env/*.
• —LeHome / Isaac Lab: the external NVIDIA/Ubuntu route for actual bimanual garment data generation and ACT, Diffusion, or SmolVLA training.

The browser workbench provides:

• —twelve virtual joint controls with actual and target positions;
• —can/cube scene placement, browser cloth, and overhead/oblique cameras;
• —Home, Wide, Reach, and Lift target poses;
• —0.5–3.5 second motion duration control;
• —quintic target interpolation with zero endpoint velocity and acceleration;
• —an emergency virtual stop and the validated cube lift-and-return replay.
• —a fixed 120 Hz XPBD towel approximation with stretch, shear, bending, table friction, point self-collision, and adjustable material sensitivity;
• —a 30 Hz browser episode recorder containing timestamps, 12D state/action, task language, object state, and cloth metrics;
• —a downloadable machine-readable training/runtime contract at /downloads/training_stack.json.

The live FPS badge reports the browser's measured render rate rather than a promised server video rate. Camera orbit, pan, and zoom are local and remain responsive even when the shared cloud sandbox is busy.

/task-space is a separate live control page for the reduced action space. It supports independent left/right XYZ control (six continuous values) and a coupled center-XYZ-plus-spread mode (four continuous values). A damped least-squares solver maps those targets to the base, shoulder, and elbow joints while both wrist joints stay fixed. Grippers remain separate discrete actions.

The two towel corners are attached by an explicit operator fixture. This is not presented as a contact grasp. Browser towel output is an inspection episode with no RGB frames and training_eligible: false; it is not a LeRobot training dataset. The exact episode contract requires overhead, left-wrist, and right-wrist RGB streams and is documented by /api/training-stack.

The Virtual Playground and Model API share the same cloud MuJoCo instance, but neither one imports LeRobot hardware drivers or writes to a physical robot.

Online model API

/model-lab exposes synchronous model-facing rigid-object environment API v2:

• —GET /api/env/spec describes the 12D joint action and observation contract.
• —POST /api/env/reset starts a can or cube episode at a selected table preset.
• —GET /api/env/observation returns actual joint state, target state, object pose, reward, termination flags, and the current RGB-frame URL.
• —POST /api/env/step applies one 12D absolute or delta joint action and returns only after the requested MuJoCo control interval has completed.
• —/downloads/so101_env_client.py is a dependency-free Python client and smoke test for the hosted Space.

The hosted Space is a shared CPU sandbox. It is suitable for API integration, environment checks, baseline rollouts, and dataset plumbing. It is not a GPU inference host. X-VLA SoftFold's 20D dual end-effector output is deliberately not passed directly into this 12D joint environment; an IK, normalization, and safety adapter is required first.

What is real

• —Two pinned LeRobot episodes are packaged locally with their real overhead videos: Qiu-Xinchuan episode 175 (778 samples) and ffifdyop episode 47 (2,008 samples).
• —The replay uses observation.state, the joint position the follower arms actually reached. The original action remains in each NPZ for later policy training and action/state error analysis.
• —The robot model is Google DeepMind MuJoCo Menagerie's SO-101 model, pinned at commit 71f066ad0be9cd271f7ed58c030243ef157af9f4.
• —During replay only the twelve force-limited actuators receive targets. No can, cube, towel body, or towel vertex is moved directly.

What is fitted rather than measured

• —Two arm mounts and the overhead camera have not been surveyed.
• —The current towel is an 81.6 x 45 cm grid. Its width and yaw are inferred from episode-175 closing-jaw locations; depth is estimated from the real video.
• —The 180 g mass, 3 mm effective thickness, 5 kPa bend modulus, and friction 1.0 are bracket values. They are not measurements of the source towel.
• —The model uses the stock SO-101 jaw collision mesh. No imaginary fingertip pad is added. The official project documents optional TPU 95A fingers and 3M grip material, but those must be modeled and measured before use here.

Current towel verdict: FAIL

validate_real_towel_replay.py runs all 25.9 seconds of episode 175 through MuJoCo at a 1 ms physics step. On the current scene:

• —numerical state remains finite with zero resets;
• —a cloth corner reaches 7.135 cm above the tabletop;
• —at most 19.44% of vertices are more than 3 cm above the table;
• —the final projected footprint changes by -7.8% (it expands rather than shrinking into a retained fold).

So the real demonstration is valid as a reference, while this MuJoCo cloth fit does not reproduce the fold. The UI and /api/physics-validation expose the same failure. The previous kinematic towel replay remains withdrawn.

The separate five-second drop/rest audit is also currently failed: it stays finite with no numerical reset, but records 5.466 mm maximum transient table penetration. Host runtime speed is reported but is not treated as a physics pass/fail criterion. The rigid baselines currently report can = fail (it falls off the table after lifting) and cube = pass under their present thresholds.

Run the audits with:

python validate_physics_gate.py
python validate_real_towel_replay.py

The second command is intentionally slow on CPU because it evaluates 25,901 contact steps rather than rendering a fake animation.

Next high-fidelity route

For cloth training, use this Space for joint mapping and failure diagnosis, then move the deformable-object work to NVIDIA Isaac Sim using the official SO-ARM starter plus the open LeHome/GarmentLab cloth stacks. Exact sources, pinned dataset commits, integration status, and WowRobo hardware facts are recorded in `data/source_registry.json` and `OPEN_SOURCE_REFERENCES.md`.

The public runtime never enumerates serial ports and always reports hardware_writes: false and real_robot_connected: false.

Local calibration bridge

The Space also exposes /calibration, a local-first interface for the separate WOWROBO calibration service. The hosted container still never opens serial ports. A visitor must explicitly connect the page to http://127.0.0.1:8771 in their own browser; without that local service the page remains offline. The local service restricts cross-origin access to this Space origin and retains all calibration files on the user's Mac.

Live joint monitor

/monitor uses the same explicit local bridge to show read-only telemetry for all twelve joints. It reports raw and calibrated position, velocity, load, current, temperature, voltage, torque enable, moving state, status code, and the calibration offset/range for each motor. Connecting an arm verifies that all six motors have torque disabled before polling; the page has no motion or calibration-write controls. Unresponsive motors and partial arms remain visibly offline.