RLWRLD/RLDX-1-PT-IMG

RLDX-1

Paper  ·  Project page  ·  Code  ·  Models

<p align="center"> <img src="teaser.png" width="100%" alt="RLDX-1 teaser"> </p>

RLDX-1 is a general-purpose Robot Foundation Model designed for dexterous manipulation. Powered by a Multi-Stream Action Transformer (MSAT), it seamlessly unifies multimodal perception (visual + tactile), high-DoF actuation, and memory-aware decision-making in a single architecture. RLDX-1 achieves state-of-the-art performance across diverse simulation benchmarks and is fully validated on real-world hardware.

This repository hosts `RLDX-1-PT-IMG`: a lightweight, image-input variant of the RLDX-1-PT, which uses 4-frame video inputs. This trades a minimal drop in success rate for a substantially lighter and faster policy, making it well suited to real-time and resource-constrained deployment. It is pre-trained on the same broad mixture of public manipulation corpora, providing a lightweight starting point for rapid experimentation on new embodiments and tasks.

<p align="center"> <img src="architecture.png" width="90%" alt="RLDX-1 architecture"> </p>

Highlights

• —Multi-Stream Action Transformer (MSAT). Cognition, physics, and action each get a dedicated stream coupled by joint self-attention — an extension of MM-DiT to action modeling.
• —Motion awareness. Multi-frame observations + a motion module capture temporal dynamics; intermediate VLM layers compress video tokens to keep the policy efficient.
• —Long-term memory. A memory module fuses past cognition features with the current ones for history-grounded decisions beyond a short multi-frame window.
• —Physical sensing. Tactile and torque enter as a dedicated physics stream; the decoder is jointly trained to predict future physical signals.
• —Three-stage training. Pre-training (generalization) → mid-training (functionality) → post-training (task adaptation), with synthetic data augmenting rare manipulation scenarios.
• —Real-time inference. Static graph capture + custom fused kernels bring the all-modality model to 43.7 ms / step on RTX 5090 (1.63× speedup, >22 Hz).

Released Checkpoints

This card describes RLDX-1-PT-IMG (vision foundation checkpoint). The full RLDX-1 model family:

Quick start

git clone https://github.com/RLWRLD/RLDX-1.git
cd RLDX
uv sync --python 3.10
uv pip install -e .

Inference (single step)

from rldx.policy.rldx_policy import RLDXPolicy
from rldx.data.embodiment_tags import EmbodimentTag

policy = RLDXPolicy(
    model_path="RLWRLD/RLDX-1-PT-IMG",
    embodiment_tag=EmbodimentTag.OXE_FRACTAL,
    device="cuda:0",
)

action = policy.get_action(observation)

RLDX-1-PT-IMG is pretrained on a multi-source mixture, so for direct inference pair it with the embodiment tag matching your data source — e.g. OXE_FRACTAL, OXE_BRIDGE_ORIG, OXE_DROID, GALAXEA, AGIBOT_GRIPPER, AGIBOT_DEXHAND, NEURAL_GR1, HUMANOID_EVERYDAY_G1, HUMANOID_EVERYDAY_H1, etc. For custom robots, finetune.

Finetune from RLDX-1-PT-IMG

uv run python rldx/experiment/launch_train.py \
    --base-model-path RLWRLD/RLDX-1-PT-IMG \
    --dataset-path /path/to/your/dataset \
    --embodiment-tag GENERAL_EMBODIMENT \
    --video-length 4 --n-cog-tokens 64 \
    --global-batch-size 64 --learning-rate 1e-4 \
    --max-steps 60000 --save-steps 5000 \
    --output-dir ./outputs/my_finetune

To enable add-ons (memory / motion / physics) see the recipes in the main README and the `training.md` guide.

Model details

• —Architecture: Multi-Stream Action Transformer (MSAT) policy with a Qwen3-VL vision-language backbone, cognition-token perceptual summary, optional Transformer memory, motion module, and tactile/torque physics encoder/decoder. Trained with flow matching.
• —Inputs: A single RGB image per camera view (video_length=1, vs the 4-frame video input of RLDX-1-PT), state proprioception, language instruction.
• —Outputs: Action chunks of length 16 (action_horizon=16).
• —Backbone: `Qwen/Qwen3-VL-8B-Instruct`.
• —Add-on modules: memory / motion / physics are dormant in this checkpoint (use_memory=use_motion=use_physics=false) and only activate when the corresponding flags are wired during finetuning (see RLDX-1-MT-ALLEX).
• —Pretraining data: A mixture of public manipulation corpora, covering Open X-Embodiment (OXE) datasets (DROID, Bridge, Fractal, Language Table, …) plus Galaxea, AgiBot World (Gripper + Dexhand), ActionNet, Neural-Curated GR-1 humanoid trajectories, and Unitree G1 / H1 from HumanoidEveryday.

For a full architectural walkthrough see `docs/architecture.md`.

Intended use & limitations

Intended use. Research on robotic manipulation, finetuning on custom embodiments, simulation benchmarking, and non-commercial real-robot deployment under the conditions of the RLWRLD Model License v1.0.

Out of scope. Commercial deployment, military or weapons applications, non-consensual surveillance, and any use that violates applicable laws or regulations. See `LICENSE.md` §3.5 for the full list.

Limitations. Performance depends heavily on embodiment match and data distribution. The pretrained checkpoint is OXE-conditioned and is not guaranteed to work zero-shot on novel embodiments without finetuning. Memory, motion, and physics modules are dormant in RLDX-1-PT-IMG and only activate when the corresponding flags are wired during finetuning (see RLDX-1-MT-ALLEX).

Citation

@article{rldx2026,
  title={RLDX-1 Technical Report},
  author={Kim, Dongyoung and Jang, Huiwon and Koo, Myungkyu and Jang, Suhyeok and Kim, Taeyoung and others},
  year={2026},
  note={RLWRLD},
  eprint={2605.03269},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2605.03269}
}

License

Released under the RLWRLD Model License v1.0 — a non-commercial license with attribution and share-alike requirements. See `LICENSE.md` for the full text. By using this model you agree to those terms, including the use restrictions in §3.5.