eugenehp/rlx-vlash

rlx-vlash weights (π₀ / π₀.₅)

RLX-native repackaging of the VLASH π₀ and π₀.₅ Vision-Language-Action robot policies, ready to load with the `rlx-vlash` crate. The tensors are the published lerobot/pi0_base / lerobot/pi05_base checkpoints, remapped to rlx-vlash's canonical key names and re-encoded into GGUF / RLX-package formats (no fine-tuning — same weights, different container + naming).

Both policies pair a PaliGemma backbone (SigLIP-So400m/14 @224 vision tower + Gemma-2B text) with a Gemma-300M action expert running through 18 joint transformer layers that share one attention, and generate action chunks by flow matching (a 10-step Euler integration of a learned velocity field).

• —π₀ — state is a suffix token, time is fused into the action embeddings, standard Gemma RMSNorm.
• —π₀.₅ — action-only suffix; time drives adaptive RMSNorm in the action expert (state_cond=false for the base checkpoint).

Contents

Each variant ships in four precisions, in both GGUF and RLX-package containers:

pi0/    model.gguf  model-q8_0.gguf  model-q4_k.gguf  model-f32.gguf
        model.rlxp  model-q8_0.rlxp  model-q4_k.rlxp  model-f32.rlxp   config.json
pi05/   (same)

Pick whichever fits: f16 is the recommended default (reproduces the action trajectory exactly), q8_0 / q4_K trade precision for size (norms/biases stay f16 in the quantized variants), f32 is bit-exact to source.

Quantization quality (important). Measured vs the original VLASH implementation on pi05_base: per-step model outputs stay near-perfect at every precision (image features & single-step velocity cosine ≥ 0.999 even at q4_K — the quant/dequant is exact), but the 10-step flow-matching rollout amplifies small weight errors into the final action chunk:

(cosine vs reference.) This is the known sensitivity of iterative flow/diffusion samplers, not a quantization bug. For faithful robot control use `f16`; treat q8_0 / q4_K as size-first options where some action drift is acceptable.

GGUF stores dims in the standard GGML (llama.cpp) order and loads through rlx-vlash or any GGUF-aware runtime; the .rlxp is the RLX flat-mmap package. All keep rlx-vlash's canonical key names, so no remap happens at load.

Usage (rlx-vlash)

Put one variant's files in a directory (the runner prefers model.gguf), then:

use rlx_vlash::{VlashRunner, VlashVariant};
use rlx_runtime::Device;

let mut runner = VlashRunner::builder(VlashVariant::Pi05)
    .device(Device::Cpu)
    .num_images(1)
    .prompt_tokens(200)
    .model_dir("path/to/pi05")   // dir containing model.gguf (+ tokenizer.json)
    .build()?;

let actions = runner.predict_action_chunk(&[image_nchw], &state, "pick up the cube", None)?;

The language tokenizer is not included (PaliGemma's is gated): fetch google/paligemma-3b-pt-224's tokenizer.json and place it alongside the weights, or supply token ids directly.

Parity

rlx-vlash reproduces the original VLASH implementation on these checkpoints at cosine = 1.000000 across every stage — SigLIP vision features, the image+text prefix, a single flow-matching denoise step (joint attention + adaRMS / state-token path), and the full 10-step action chunk — for both π₀ and π₀.₅ (max |Δ| ≤ 1.1e-3, attributable to MPS-f32 reference vs CPU-f32 rlx).

License & attribution

Apache-2.0, inherited from the upstream VLASH project and the lerobot/pi0_base / lerobot/pi05_base base checkpoints. These files are a format/naming repackaging of those weights; please cite VLASH and LeRobot. The rlx-vlash inference crate itself is GPL-3.0-only.