vrfai/pi05-libero-gguf

π0.5 (pi05) - LIBERO base (GGUF for vla.cpp)

GGUF conversion of `lerobot/pi05_libero_base` for inference with **vla.cpp**, a lightweight C++ inference engine for Vision-Language-Action models built on top of `llama.cpp`.

π0.5 pairs a PaliGemma vision-language backbone (gemma_2b variant) with a Gemma-300m action expert running a flow-matching head. It differs from π0 in three ways that the vla.cpp runtime reproduces exactly: /

• —State-in-prompt - the proprioceptive state is per-dimension quantile-normalised, discretised into 256 bins, and appended to the text prompt (Task: <instruction>, State: <bins>;\nAction: ) instead of being fed as a state token.
• —adaRMS action expert - each expert block uses adaptive RMS norm: a dense(cond) projection of the flow-matching timestep embedding produces per-layer scale/shift/gate, applied to gated residuals.
• —Quantile action normalisation - actions are un-normalised with q01/q99 quantiles (baked into the GGUF) rather than mean/std.

Usage

# Terminal 1 - serve (use the CUDA build for inference)
./build-cuda/vla-server --bind tcp://*:5555 \
    pi05-libero.gguf

# Terminal 2 - drive a LIBERO episode (inside the LIBERO uv venv)
python eval/client/run_sim_client_direct.py \
    --arch pi05 \
    --task libero_object --task-id 0 --n-episodes 10 \
    --n-action-steps 10 \
    --max-length 200 \
    --stats-json <LIBERO meta/stats.json> \
    --vla-addr tcp://localhost:5555

Notes:

• —π0.5 uses the gated google/paligemma-3b-pt-224 tokenizer. Run huggingface-cli login and accept the licence, or point --tokenizer at a local copy.
• —--stats-json must point at a LIBERO meta/stats.json that carries q01/q99 for observation.state (π0.5 discretises the state with quantiles). If omitted, the client auto-downloads lerobot/libero meta/stats.json.
• —--n-action-steps 10 matches the checkpoint's n_action_steps (chunk size 50, 10 Euler denoise steps). --max-length 200 matches the training prompt budget.

Validation

Runtime parity (vs the lerobot PI05Pytorch.sample_actions reference, F32 both sides, image embeddings fed in to isolate the action stack): worst |Δ| = 1.98e-4 over the full prefill + 10-step Euler denoise - well within the 2e-3 threshold.

LIBERO smoke test (libero_object / task 0, 10 episodes, RTX 3090):

This is a single-task smoke run to confirm the end-to-end pipeline, not the full libero_object sweep (10 tasks × N episodes).

License

Weights follow the upstream license of `lerobot/pi05_libero_base` (Apache-2.0). The vla.cpp conversion tooling and inference engine are MIT-licensed.