RoboDyna/X-VLA-RoboDyna

X-VLA-RoboDyna

X-VLA-0.9B fine-tuned on RoboDyna Benchmark v2 — a dual-arm (ur5-wsg) benchmark of dynamic manipulation: moving targets, rolling and falling objects, closing time windows, conveyor belts and distractors.

Trained on all 4,050 episodes / 35 tasks (1.37M frames, 16.67 Hz, three RGB views).

Status

No rollout evaluation has been run yet — this card reports no success rate. What is verified is training convergence and that the checkpoint loads and is self-contained. Treat it as a trained baseline, not a benchmarked result.

Usage

from transformers import AutoModel, AutoProcessor

model     = AutoModel.from_pretrained("RoboDyna/X-VLA-RoboDyna", trust_remote_code=True)
processor = AutoProcessor.from_pretrained("RoboDyna/X-VLA-RoboDyna", trust_remote_code=True)

# X-VLA serves over HTTP; see the upstream X-VLA repo for the client side.
model.run(processor, host="0.0.0.0", port=8000)

Pass domain_id = 19 — the soft-prompt slot this model was trained under (see below).

Action space — read this before deploying

The model emits X-VLA's 20-D ee6d vector, absolute end-effector pose (no deltas), laid out as:

[0:3]   arm0 xyz        [3:9]   arm0 rot6d      [9]   arm0 gripper
[10:13] arm1 xyz        [13:19] arm1 rot6d      [19]  arm1 gripper

arm0 = left, arm1 = right. Conventions that must match on the decode side:

• —rot6d is the first two columns of the rotation matrix, flattened row-major, i.e. interleaved: c1 = v[0::2], c2 = v[1::2]. Slicing it as v[0:3]/v[3:6] is wrong.
• —Gripper is 1 = closed (X-VLA convention). RoboDyna's raw observation.endpose gripper is 1 = open, so it was inverted during training; invert back when replaying against the dataset. Gripper channels are logits — apply sigmoid (the model's postprocess does this).
• —Source quaternions in observation.endpose are scalar-first (wxyz), per the SAPIEN/RoboTwin generator (transforms3d.mat2quat). This was confirmed against the dataset's own recorded wrist-camera extrinsics: for the correct order, R_cam(t)^T @ R_endpose(t) is constant to 0.013 deg, versus 4.64 deg for scalar-last.

The action chunk is 30 steps sampled at the native 16.67 Hz control rate (~1.8 s).

Note the RoboDyna dataset's action column is 14-D joint space; this model does not use it. X-VLA's EE6DActionSpace hard-codes its channel indices, so joint vectors cannot be padded into the 20-D slot — the model is trained from observation.endpose instead.

Training

Loss (median per 5k block): 1.466 → 0.258 → 0.182 → 0.134 → 0.110 → 0.114. Final 2k steps: median 0.105. Flat over the last two blocks, i.e. converged within the 30k budget.

The soft-prompt bank moved only at index 19 (drift 1.4e-3); domains 0–18 are bit-identical to the base checkpoint, so pretrained cross-embodiment prompts are uncontaminated.

Contents

Includes the trust_remote_code modeling files (modeling_xvla.py, configuration_xvla.py, transformer.py, action_hub.py, processing_xvla.py, and the Florence-2 pair), so the repo loads standalone.

Citation

@article{zheng2025x,
  title   = {X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Model},
  author  = {Zheng, Jinliang and Li, Jianxiong and Wang, Zhihao and Liu, Dongxiu and Kang, Xirui
             and Feng, Yuchun and Zheng, Yinan and Zou, Jiayin and Chen, Yilun and Zeng, Jia and others},
  journal = {arXiv preprint arXiv:2510.10274},
  year    = {2025}
}