Kouch/SO101_IsaacSim_Test_V3

1---2datasets: Kouch/SO101_IsaacSim_Block_Pick_and_Place_V23library_name: lerobot4license: apache-2.05model_name: act6pipeline_tag: robotics7tags:8- lerobot9- robotics10- act11---12 13# Model Card for act14 15<!-- Provide a quick summary of what the model is/does. -->16 17 18[Action Chunking with Transformers (ACT)](https://huggingface.co/papers/2304.13705) is an imitation-learning method that predicts short action chunks instead of single steps. It learns from teleoperated data and often achieves high success rates.19 20 21 22 23<!-- A short demo is worth more than any description! Record a GIF/video of the policy24running on your robot, upload it to this repo, and embed it here:25<p align="center">26  <img src="https://huggingface.co/<hf_user>/<policy_repo_id>/resolve/main/demo.gif" width="60%"/>27</p>28-->29 30This policy has been trained and pushed to the Hub using [LeRobot](https://github.com/huggingface/lerobot).31 32Learn how to train and run it in the [LeRobot act guide](https://huggingface.co/docs/lerobot/main/en/act), or browse the [full documentation](https://huggingface.co/docs/lerobot/index).33 34 35---36 37## Model Details38 39- **License:** apache-2.040- **Robot type:** `so101`41- **Cameras:** `top`42 43 44## Inputs & Outputs45 46The policy consumes these observation features and produces these action features.47 48**Inputs**49 50| Feature | Type | Shape |51| --- | --- | --- |52| `observation.state` | STATE | `(6,)` |53| `observation.images.top` | VISUAL | `(3, 480, 640)` |54 55**Outputs**56 57| Feature | Type | Shape |58| --- | --- | --- |59| `action` | ACTION | `(6,)` |60 61 62## Training Dataset63 64- **Repository:** [Kouch/SO101_IsaacSim_Block_Pick_and_Place_V2](https://huggingface.co/datasets/Kouch/SO101_IsaacSim_Block_Pick_and_Place_V2)65- **Episodes:** 2066- **Frames:** 808367- **Frame rate:** 30 FPS68- **Task(s):** "Pick up blue cube and place it to the box"69 70<a class="flex" href="https://huggingface.co/spaces/lerobot/visualize_dataset?path=Kouch/SO101_IsaacSim_Block_Pick_and_Place_V2">71<img class="block dark:hidden" src="https://huggingface.co/datasets/huggingface/badges/resolve/main/visualize-this-dataset-xl.svg"/>72<img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface/badges/resolve/main/visualize-this-dataset-xl-dark.svg"/>73</a>74 75 76## Training Configuration77 78| Setting | Value |79| --- | --- |80| Training steps | 100000 |81| Batch size | 8 |82| Optimizer | adamw |83| Learning rate | 1e-05 |84| Seed | 1000 |85| LeRobot version | 0.6.2 |86 87---88 89## How to Get Started with the Model90 91New to LeRobot? These guides cover the full workflow:92 93- **[Install LeRobot](https://huggingface.co/docs/lerobot/main/en/installation)** — set up the `lerobot` package.94- **[Hardware setup](https://huggingface.co/docs/lerobot/main/en/hardware_guide)** — assemble, wire, and calibrate your robot and cameras.95- **[Record data & train a policy](https://huggingface.co/docs/lerobot/en/il_robots)** — the end-to-end imitation-learning walkthrough.96- **[CLI cheat-sheet](https://huggingface.co/docs/lerobot/main/en/cheat-sheet)** — quick reference for the `lerobot-*` commands.97 98The short version to run and train this policy:99 100### Run the policy on your robot101 102```bash103lerobot-rollout \104  --strategy.type=base \105  --robot.type=so101 \106  --robot.port=<your_robot_port> \107  --robot.cameras="{ <camera_1>: {type: opencv, index_or_path: <index_or_path>, width: 640, height: 480, fps: 30}, <camera_2>: {type: opencv, index_or_path: <index_or_path>, width: 640, height: 480, fps: 30}}" \108  --policy.path=Kouch/SO101_IsaacSim_Test_V3 \109  --task="Pick up blue cube and place it to the box" \110  --duration=60111```112 113Replace the remaining `<...>` placeholders with your own values: `--robot.port` and the camera names/indices are specific to your machine, and the camera names must match the observation keys this policy was trained on.114 115When `--strategy.type=base` is used the script doesn't record the episodes. Skipping duration will make the policy run indefinitely. For more information look at [rollout documentation](https://huggingface.co/docs/lerobot/main/en/inference).116 117### Train your own policy118 119```bash120lerobot-train \121  --dataset.repo_id=${HF_USER}/<dataset> \122  --policy.type=act \123  --output_dir=outputs/train/<policy_repo_id> \124  --job_name=lerobot_training \125  --policy.device=cuda \126  --policy.repo_id=${HF_USER}/<policy_repo_id> \127  --wandb.enable=true128```129 130_Writes checkpoints to `outputs/train/<policy_repo_id>/checkpoints/`._131 132---133 134## Evaluation135 136<!-- Report real-robot results here: run the policy several times per task and count the137successes. Delete the "No evaluation results" line and fill in this table instead:138 139| Task | Trials | Successes | Success rate |140| ---- | ------ | --------- | ------------ |141| pick the lego brick | 10 | 8 | 80% |142 143Also worth noting: anything that affects difficulty (new object positions, lighting,144distractors, a different robot of the same type, ...).145-->146 147_No evaluation results have been provided for this policy yet._148 149---150 151## Citation152 153If you use this policy, please cite the method linked in the description above, along with LeRobot:154 155```bibtex156@misc{cadene2024lerobot,157    author = {Cadene, Remi and Alibert, Simon and Soare, Alexander and Gallouedec, Quentin and Zouitine, Adil and Palma, Steven and Kooijmans, Pepijn and Aractingi, Michel and Shukor, Mustafa and Aubakirova, Dana and Russi, Martino and Capuano, Francesco and Pascal, Caroline and Choghari, Jade and Moss, Jess and Wolf, Thomas},158    title = {LeRobot: State-of-the-art Machine Learning for Real-World Robotics in Pytorch},159    howpublished = "\url{https://github.com/huggingface/lerobot}",160    year = {2024}161}162```