ZibinDong/fastwam_libero_uncond_2cam224

FastWAM LIBERO 2-Camera 224 (LeRobot)

FastWAM is a world action model built on Wan2.2 and an action diffusion expert. It predicts continuous robot action chunks from visual observations, proprioception, and language/task context.

Checkpoint trained and evaluated on LIBERO tasks: 97.6 % SR on all tasks.

Original paper: https://arxiv.org/pdf/2603.16666

Implementation: This LeRobot implementation follows the original reference code for compatibility. Reference implementation: [https://github.com/yuantianyuan01/FastWAM]

Model description

• —Inputs: concatenated multi-view RGB image, robot state/proprioception, task context
• —Outputs: continuous robot actions
• —Training objective: FastWAM video/action diffusion loss
• —Intended use: evaluation or fine-tuning on LIBERO-style manipulation tasks

Quick start

Installation

Install LeRobot from a version that includes the fastwam policy:

pip install "lerobot[fastwam]@git+https://github.com/huggingface/lerobot.git"

For full installation details, see the official LeRobot documentation: https://huggingface.co/docs/lerobot/installation

Load model and run select_action

import torch

from lerobot.policies.fastwam.modeling_fastwam import FastWAMPolicy

model_id = "<namespace>/fastwam-libero-uncond-2cam224"
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

policy = FastWAMPolicy.from_pretrained(model_id, strict=False).to(device).eval()

batch = {
    "observation.images.image": torch.zeros(1, 3, 224, 448, device=device),
    "observation.state": torch.zeros(1, 8, device=device),
    "prompt": "pick up the object and place it at the target location",
}

with torch.inference_mode():
    action = policy.select_action(batch)

print(action.shape)

FastWAMPolicy.from_pretrained(...) loads the policy weights and the local Wan sidecar components from this same repository snapshot.

Training step

For training or fine-tuning, call forward(...) and use the returned loss key:

policy.train()

outputs = policy.forward(batch)
loss = outputs["loss"]
loss.backward()

The training batch must contain FastWAM-ready tensors such as video, action, context, and context_mask, or LeRobot observation/action keys that can be adapted by the policy wrapper.

Fine-tuning

A typical fine-tuning command follows the standard LeRobot training flow:

lerobot-train \
  --dataset.repo_id=<your-libero-style-dataset> \
  --output_dir=./outputs/fastwam_finetune \
  --job_name=fastwam_finetune \
  --policy.type=fastwam \
  --policy.path=<namespace>/fastwam-libero-uncond-2cam224 \
  --policy.device=cuda \
  --steps=100000 \
  --batch_size=1

Adjust batch size and sequence settings for available GPU memory.

Evaluate in simulation

For LIBERO evaluation, use the LeRobot evaluation flow once your environment is installed:

lerobot-eval \
  --policy.path=<namespace>/fastwam-libero-uncond-2cam224 \
  --env.type=libero \
  --env.task=libero_spatial \
  --eval.batch_size=1 \
  --eval.n_episodes=20

If using a project-specific LIBERO evaluator, pass this repository id as the policy path.

Repository files

This repository is self-contained for FastWAMPolicy.from_pretrained(...):

config.json
model.safetensors
policy_preprocessor.json
policy_preprocessor_step_3_normalizer_processor.safetensors
policy_postprocessor.json
policy_postprocessor_step_0_unnormalizer_processor.safetensors
Wan2.2_VAE.safetensors
models_t5_umt5-xxl-enc-bf16.safetensors
google/umt5-xxl/
libero_uncond_2cam224_dataset_stats.json

The Wan VAE, UMT5 text encoder, and tokenizer are stored beside the FastWAM policy weights. The saved policy postprocessor also sets toggle_action_dimensions: [-1] to match the LeRobot LIBERO gripper action convention.

Notes

This checkpoint uses only the migrated Hugging Face / LeRobot serialization format: config.json, model.safetensors, and local Wan sidecar files. Original FastWAM .pt checkpoint loading is not required.