OpenRAL/rskill-xr1-franka_panda-vlabench-nf4

XR-1 VLABench

OpenRAL wrapper for Xiaomi Robotics' XR-1 VLABench checkpoint on the Franka Panda benchmark embodiment.

Preview

Clean VLABench rollout (fail)

What this skill does

How it works

The sidecar receives VLABench's front, second, and wrist views at 480x480 and the raw 7-D relative EE state. XR-1 predicts ten 7-D deltas. OpenRAL integrates those deltas into the absolute targets consumed by VLABench and executes five before replanning. The gripper scalar uses Xiaomi's 0.2 threshold and maps to OpenRAL's normalized closed/open commands.

Camera order is load-bearing and matches Xiaomi's evaluator: camera1=front(raw index 2), camera2=base/second(raw index 0), camera3=wrist(raw index 3). Frames are not flipped (flip_180: false).

Observation -> action contract

Upstream model and training

Supported robots

Sensors required

Three RGB streams at 480x480 and VLABench's 7-D EE/gripper state.

Manifest summary

Quick start

just sync --group sidecar-wire
OPENRAL_ALLOW_REMOTE_CODE=1 openral sim run \
  --config scenes/benchmark/vlabench_select_fruit.yaml \
  --rskill rskills/xr1-vlabench

Reproduction

Provision the VLABench asset bundle documented by the scene backend, then run the command above. This package ships no score until that real simulator run has completed.

Persist NF4 locally

The default sidecar quantizes the pinned BF16 source at load. Export once to avoid that repeated pack step:

OPENRAL_ALLOW_REMOTE_CODE=1 uv run python tools/xr1_sidecar.py \
  --model hf://XiaomiRobotics/Xiaomi-Robotics-1-VLABench@2dfc33b390478f71737eacb4748333e6d8638a06 \
  --profile vlabench_choice --quantization nf4 \
  --export-dir outputs/run_artifacts/xr1-vlabench-nf4-rskill/weights

The local unpublished package is then outputs/run_artifacts/xr1-vlabench-nf4-rskill. Its packed output was verified bit-identical to runtime NF4 on the same seeded input (MAE=0, max error=0). On the reference host, policy initialization dropped from about 18.0 s to 10.1 s.

Evaluation

NF4 was loaded and executed on an RTX 4070 Laptop 8 GB against the real VLABench select_fruit environment. The sidecar used 3.66 GiB process VRAM; the first synthetic chunk took 1.35 s and the warmed real-scene chunk took 0.81 s. One finite (10, 7) chunk was integrated and applied successfully. This validates execution, not task success or Xiaomi's benchmark score.

License

The wrapper and upstream code/weights are Apache-2.0. Loading requires OPENRAL_ALLOW_REMOTE_CODE=1 for the checkpoint's custom Transformers code.

See also

• —`scenes/benchmark/vlabench_select_fruit.yaml`
• —`docs/reference/vla_compatibility.md`