Modelpublic

ParsaSharifi/patch-policy-libero-spatial-diffusion

sourceHugging Facemitupdated 6h agoView on Hugging Face

0likes

Model Card

Patch Policy diffusion on LIBERO Spatial

A Patch Policy diffusion head trained on LIBERO Spatial: frozen DINOv2 ViT-S/14 patch tokens from both cameras (512 tokens), conditioned on a per-task goal image, predicting 7-d end-effector actions. Checkpoint of epoch 122, the best of a 200-epoch run after a 500-episode re-evaluation.

Result

checkpoint	success	episodes
epoch 122 (this checkpoint)	0.944	500 (50 per task)
epoch 122	0.99	100 (10 per task)

Per task, epoch 122, 500 episodes:

task: pick up the black bowl ...	success
between the plate and the ramekin	0.88
from table center	1.00
in the top drawer of the wooden cabinet	0.98
next to the cookie box	1.00
next to the plate	0.80
next to the ramekin	0.98
on the cookie box	0.96
on the ramekin	0.96
on the stove	0.98
on the wooden cabinet	0.90

—Evaluation protocol of the patch-policy code, not the LIBERO standard: a fresh seeded reset per episode (seed 42 + k, objects jittered within the task's regions), 300-step limit. The LIBERO standard uses 50 fixed initial states per task and 220 steps, so this number is not directly comparable to published LIBERO Spatial results.
—All sweep and confirmation results are in results/ (summary.json, eval/model_<epoch>.json, eval/confirm/). The 10 goal images are in results/goal_images/.

Training

setting	value
code	patchpolicycleanup, branch `feat/libero-spatial` (commit `7fda225`), config `libero_spatial/diffusion`
data	ParsaSharifi/patch-policy-libero-spatial: the 500 official LIBERO Spatial demos re-rendered at 224x224
observation / action windows	2 / 3 (prediction horizon 4)
batch, epochs, optimiser	255 (85 x 3 GPUs), 200, AdamW lr 5e-5, weight decay 2e-4
denoiser	transformer 8 layers x 4 heads x 256; DDPM 100 steps, EMA weights used for evaluation
precision	bf16 autocast, torch.compile, TF32 (DINOv2 runs in fp32)
goal	last frame of each task's first-listed demo (`demo_36`), both cameras, concatenated per token
hardware, time	3 x RTX 3090, 4 h

Files

outputs/libero_spatial/diffusion/dino_patch-2026-09-21_212317/checkpoints/model_0122.pt   # epoch, config, policy_state_dict
results/                                                                                 # evaluation results, goal images, frame strips

The checkpoint is a PyTorch pickle (torch.load(..., weights_only=False)): load it only if you trust this repo. DINOv2 is not included; the code loads it from torch.hub (facebookresearch/dinov2).

Evaluate

From a checkout of the code on branch feat/libero-spatial, with the dataset downloaded:

bash

hf download ParsaSharifi/patch-policy-libero-spatial-diffusion --local-dir ~/pp_libero_spatial
MUJOCO_GL=egl uv run python evaluate.py \
  ~/pp_libero_spatial/outputs/libero_spatial/diffusion/dino_patch-2026-09-21_212317 --epochs 122 --no-wandb

—The config stored in the checkpoint names the dataset directory it was trained with (dataset.data_directory: /media/sda1/parsa/original_patch_policy/patch_policy_datasets/libero_dataset). The evaluation loads the goal images from there, so the dataset must be at that path (or a symlink to it).
—The goal image of each task is the last frame of the first demo its folder lists. On the training machine that is demo_36 for every task; directory listing order depends on the filesystem, so check it on a new machine.