Apppublic

hugging-apps/flex-pi-libero-demo

sourceHugging Facemitupdated 3d agoView on Hugging Face

0likes

App README

Flex-π on LIBERO

Closed-loop rollouts of Flex-π, the multi-stream world-action model from *Flex-π: A Multi-Stream World-Action Model with Compute Flexibility* (project page · code), driving a real LIBERO MuJoCo simulation.

Pick one of the 40 LIBERO tasks, hit Run rollout, and the Space:

1.builds the task's MuJoCo environment and resets it to one of the 50 saved init states,
2.every replan steps feeds the live agentview + wrist RGB (plus the simulator's depth, turned into a normalized XYZ pointmap) and the language instruction into the policy,
3.executes the returned 32-step action chunk in the sim,
4.streams frames back and, at the end, renders the rollout video, the model's own predicted future video (the world-model stream), and the action trajectory it committed to.

The single checkpoint supports 56 deploy regimes. Three are exposed here:

Regime	Generated streams	Cost
Action only	action chunk (KV-cache fast path)	cheapest
Action + future video	action chunk + RGB future	medium
Action + video + DINO + pointmap	all four streams jointly	most expensive, best 3D grounding

That is the paper's point: you trade denoising compute for grounding at inference time, without retraining.

—ZeroGPU gives each call a bounded GPU slice, so the step budget is capped well below LIBERO's official 400/700-step limits — a failed rollout here is not a benchmark number.
—Depth comes from the simulator (as in the paper's eval), so this is a simulator demo rather than a bring-your-own-photo demo; the instruction textbox is free-text, and the T5 encoder is loaded so you can rewrite it.
—MuJoCo renders through OSMesa on CPU; the GPU is used only for the policy.

—Flex-π: model, src/flexpi, and experiments/libero are vendored from geyan21/flex-pi (MIT, see LICENSE-flexpi).
—LIBERO: third_party/LIBERO is vendored from Lifelong-Robot-Learning/LIBERO (MIT, see third_party/LIBERO/LICENSE).