hugging-apps/pwm-wrop-object-permanence

PWM-WROP — a world model that keeps objects real

`Hokin/PWM-WROP` is a 15.6B parameter video world model: `nvidia/Cosmos3-Nano` fine-tuned on WROP (World Representation of Object Permanence). Feed it a short clip plus a prompt; it conditions on the last 57 frames (~2.4 s at 24 fps) and predicts the next 60 frames at 320×192.

The demo runs the authors' own inference stack, vendored from hokindeng/object-permanence (pwm/) and ported from AWS Trainium to CUDA:

• —geometry and sampler settings straight from pwm/configs/wrop.yaml — 320×192, 24 fps, latent_t 30 (15 clean + 15 predicted latent frames), text_len 128, 35 UniPC steps, guidance 6.0, shift 10.0;
• —prompt packing via pwm.data.pack (chat template + eos + <|vision_start|>, right-padded to 128 with the pads hidden from every vision row);
• —conditioning latents from the Wan2.2 VAE (nvidia/Cosmos3-Nano vae/), UniPC flow sampling with classifier-free guidance, conditioning frames held bit-exact throughout.

The only deliberate deviation from the reference implementation: two-way attention uses F.scaled_dot_product_attention with an explicit boolean mask instead of the repo's explicit-fp32 kernels (those exist because a masked SDPA backward is imprecise on Neuron; this demo is forward-only on CUDA). The fast path is checked numerically against the original at startup and falls back automatically if it ever disagrees.

Examples

The four example clips and prompts come from `Hokin/object-permanence-benchmark` (eval/shard{1,2}/…/input_video.mp4 + prompt.txt, used verbatim), CC BY-NC 4.0 © Hokin Deng.

License

• —pwm/ and the example assets: CC BY-NC 4.0, © 2026 Hokin Deng (see LICENSE_object_permanence).
• —Base model nvidia/Cosmos3-Nano: NVIDIA Cosmos OpenMDW-1.1.

Non-commercial use only.