CoolFace
Apppublic

Camais03/Camie-Crafter

sourceHugging Faceupdated 6mo agoView on Hugging Face
3likes
App README

Camie Crafter World Model

Interactive demo for the Camie Crafter latent world model.

This Space loads a real Crafter clip, encodes the first 32 frames into latent context with an MAE encoder, then rolls the world model forward one action at a time and decodes each predicted latent back into an image.

This is not the real Crafter environment. It is the model’s imagined continuation.

What the app does

At startup the Space downloads and loads these ONNX exports from the repo:

  • —checkpoints/world_model_one_step.onnx
  • —checkpoints/mae_decode_one_frame.onnx
  • —checkpoints/mae_encode.onnx

Then the UI lets you:

  • —load a built-in demo .npz clip
  • —upload your own Crafter .npz episode
  • —choose a starting frame index
  • —encode the context window into latents
  • —step the model forward with discrete Crafter actions
  • —save the imagined trajectory as a GIF

Model pipeline

For each rollout:

  1. 1.Encode context The selected context frames are passed through mae_encode.onnx.
  1. 1.Predict next latent The encoded context latents and chosen action are passed into world_model_one_step.onnx.
  1. 1.Decode predicted frame The predicted latent is rendered with mae_decode_one_frame.onnx.
  1. 1.Feed prediction back in The predicted latent is appended back into context for open-loop imagined play.

Controls

Movement

  • —W / ArrowUp → up
  • —A / ArrowLeft → left
  • —S / ArrowDown → down
  • —D / ArrowRight → right

Other actions

  • —Space → do
  • —Tab → noop
  • —Shift → sleep

Craft / place

  • —1 → place stone
  • —2 → place table
  • —3 → place furnace
  • —4 → place plant
  • —5 → make wood pickaxe
  • —6 → make stone pickaxe
  • —7 → make iron pickaxe
  • —8 → make wood sword
  • —9 → make stone sword
  • —0 → make iron sword

Input format

The app expects a Crafter .npz file containing at least:

  • —image: array of frames with shape [T, H, W, C]
  • —optionally action: array of integer actions with shape [T]

Expected details

  • —frames should be RGB uint8
  • —resolution should match the model export, typically 64×64
  • —if action is missing, the app fills actions with zeros

Built-in demo clip

By default the app loads the demo file:

  • —demo/1CKFwmfLeb5MzlgFRaIF7M.npz

You can change this with:

  • —CAMIE_DEMO_NPZ

Environment variables

Optional configuration:

  • —CAMIE_CHECKPOINT_SUBDIR default: checkpoints
  • —CAMIE_DEMO_NPZ default: demo/1CKFwmfLeb5MzlgFRaIF7M.npz
  • —CAMIE_N_CTX default: 32
  • —CAMIE_IMG_SIZE default: 64
  • —CAMIE_MODEL_NUM_LATENTS default: 32
  • —CAMIE_MODEL_D_LAT default: 32
  • —CAMIE_SEED default: 0

Notes

  • —The app runs entirely with ONNX Runtime.
  • —The first load may take a moment because the ONNX files are downloaded at startup.
  • —world_model_one_step.onnx may not have a separate .data sidecar file, which is fine.
  • —Uploaded clips are used only as starting context; after that, the rollout is purely model-generated.

Requirements

Listed in requirements.txt:

  • —gradio
  • —onnx
  • —onnxruntime
  • —huggingface_hub
  • —numpy
  • —pillow

Local run

bash
python app.py