hugging-apps/orca-4b-world-model

Orca-4B — World Model (Visual QA)

Interactive demo of **BAAI/Orca-4B**, a general world foundation model centered on Next-State-Prediction. Orca learns a unified world latent space from multimodal signals (video + language) and exposes it through lightweight, frozen-backbone readout decoders for text, images, and actions.

This Space exposes two of Orca's readouts:

• —Scene / Next-State QA (text readout): given an image (the current world state) and a question, the model reasons about the scene's dynamics and likely next state. This mirrors the authors' released text-generation evaluation code, which loads the VLM readout from the bundled checkpoint (vlm.model.* weights over a Qwen3.5-VL backbone).
• —Next-State Prediction (image + action readout heads): given an image and an action / event description, the model runs the bundled Next-State-Prediction heads (nfp_head for the short-horizon next state, long_event_head for the long horizon) over the learnable query tokens conditioned on the world latent, then renders the predicted next-state world latent as a spatial latent map and reports the action readout (the latent projected onto the policy action space defined by policy_preprocessor.json).

Runs on ZeroGPU. The released checkpoint ships the latent next-state-prediction heads; the separate pixel/robot decoders are not part of this checkpoint, so the predicted next-state latent is visualized directly.