lerobot/diffusion_pusht

1---2license: apache-2.03datasets:4- lerobot/pusht5tags:6- diffusion-policy7- model_hub_mixin8- pytorch_model_hub_mixin9- robotics10pipeline_tag: robotics11---12# Model Card for Diffusion Policy / PushT13 14Diffusion Policy (as per [Diffusion Policy: Visuomotor Policy15Learning via Action Diffusion](https://arxiv.org/abs/2303.04137)) trained for the `PushT` environment from [gym-pusht](https://github.com/huggingface/gym-pusht).16 17## How to Get Started with the Model18 19See the [LeRobot library](https://github.com/huggingface/lerobot) (particularly the [evaluation script](https://github.com/huggingface/lerobot/blob/main/lerobot/scripts/eval.py)) for instructions on how to load and evaluate this model.20 21## Training Details22 23Trained with [LeRobot@3c0a209](https://github.com/huggingface/lerobot/tree/3c0a209f9fac4d2a57617e686a7f2a2309144ba2).24 25The model was trained using [LeRobot's training script](https://github.com/huggingface/lerobot/blob/main/lerobot/scripts/train.py) and with the [pusht](https://huggingface.co/datasets/lerobot/pusht) dataset, using this command:26 27```bash28python lerobot/scripts/train.py \29    --output_dir=outputs/train/diffusion_pusht \30    --policy.type=diffusion \31    --dataset.repo_id=lerobot/pusht \32    --seed=100000 \33    --env.type=pusht \34    --batch_size=64 \35    --steps=200000 \36    --eval_freq=25000 \37    --save_freq=25000 \38    --wandb.enable=true39```40 41 42The training curves may be found at https://wandb.ai/aliberts/lerobot/runs/s7elvf4r.43The current model corresponds to the checkpoint at 175k steps.44 45## Evaluation46 47The model was evaluated on the `PushT` environment from [gym-pusht](https://github.com/huggingface/gym-pusht) and compared to a similar model trained with the original [Diffusion Policy code](https://github.com/real-stanford/diffusion_policy). There are two evaluation metrics on a per-episode basis:48 49- Maximum overlap with target (seen as `eval/avg_max_reward` in the charts above). This ranges in [0, 1].50- Success: whether or not the maximum overlap is at least 95%.51 52Here are the metrics for 500 episodes worth of evaluation. The "Theirs" column is for an equivalent model trained on the original Diffusion Policy repository and evaluated on LeRobot (the model weights may be found in the [`original_dp_repo`](https://huggingface.co/lerobot/diffusion_pusht/tree/original_dp_repo) branch of this respository).53 54<blank>|Ours|Theirs55-|-|-56Average max. overlap ratio | 0.955 | 0.95757Success rate for 500 episodes (%) | 65.4 | 64.258 59The results of each of the individual rollouts may be found in [eval_info.json](eval_info.json).60It was produced after training with this command:61```bash62python lerobot/scripts/eval.py \63    --policy.path=outputs/train/diffusion_pusht/checkpoints/175000/pretrained_model \64    --output_dir=outputs/eval/diffusion_pusht/175000 \65    --env.type=pusht \66    --eval.n_episodes=500 \67    --eval.batch_size=50 \68    --device=cuda \69    --use_amp=false70```