Datasetpublic

yqi19/omnireset_reproduce

OmniReset → GR00T — data-generation code & instructions This repo contains the code and instructions to generate a GR00T-flavored LeRobot dataset of UR5e + Robotiq-2F85 peg-insertion demonstrations, by rolling out the OmniReset state-based RL expert (oracle) policy in Isaac Sim. ⚠️ The dataset itself is not hosted here — this repo is code + docs only. Follow the steps below to generate the data yourself (it's fast: ~20 demos/min on one A40 after the fixes). Repo… See the full description on the dataset page: https://huggingface.co/datasets/yqi19/omnireset_reproduce.

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face

0likes25downloads

Dataset Card

OmniReset → GR00T — data-generation code & instructions

This repo contains the code and instructions to generate a GR00T-flavored LeRobot dataset of UR5e + Robotiq-2F85 peg-insertion demonstrations, by rolling out the OmniReset state-based RL expert (oracle) policy in Isaac Sim.

⚠️ The dataset itself is not hosted here — this repo is code + docs only. Follow the steps below to generate the data yourself (it's fast: ~20 demos/min on one A40 after the fixes).

Repo layout

.
├── code/                       # scripts + configs used to generate & convert the data
│   ├── data_collection_rgb_cfg.py   # the MODIFIED OmniReset RGB collection env config
│   ├── collect_demos.py             # roll out the expert, record episodes (+ --export_all)
│   ├── convert_to_lerobot.py        # HDF5 -> GR00T LeRobot v2.1
│   ├── omnireset_config.py          # GR00T modality config (register_modality_config)
│   ├── finetune.md                  # how to fine-tune GR00T on the generated dataset
│   └── README.md
└── docs/                       # documentation (English + 中文)
    ├── collect_demo.md              # full debugging story + goal + fixes
    ├── sample_gr00t_data.md         # sampling + camera setup + performance
    ├── omnireset_to_gr00t_format_zh.md   # 数据格式转换说明(中文)
    └── osc_action_space_zh.md            # OSC 动作空间说明(中文)

How to generate the data (3 steps)

Assumes the full UWLab / Isaac-Lab / Isaac-Sim install (conda activate env_uwlab), and the files in code/ placed at their matching UWLab paths (data_collection_rgb_cfg.py under the omnireset ur5e config; convert_to_lerobot.py / omnireset_config.py wherever convenient).

bash

export OMNI_KIT_ACCEPT_EULA=YES

# 1) download the state expert & export it to TorchScript (with compute_distribution)
wget https://huggingface.co/datasets/UW-Lab/uwlab-assets/resolve/main/Policies/OmniReset/state_based_experts/peg_state_rl_expert_seed42.pt
python scripts/reinforcement_learning/rsl_rl/play.py \
    --task OmniReset-Ur5eRobotiq2f85-RelCartesianOSC-State-Play-v0 \
    --num_envs 1 --checkpoint peg_state_rl_expert_seed42.pt --headless \
    env.scene.insertive_object=peg env.scene.receptive_object=peghole      # -> exported/policy.pt

# 2) collect 1000 successful demos (3 cameras, fixed lighting, 4 reset types uniform)
python scripts_v2/tools/collect_demos.py \
    --task OmniReset-Ur5eRobotiq2f85-RelCartesianOSC-RGB-DataCollection-v0 \
    --num_envs 20 --num_demos 1000 --deterministic --headless --enable_cameras \
    --dataset_file demos.hdf5 \
    env.scene.insertive_object=peg env.scene.receptive_object=peghole env.episode_length_s=20.0 \
    'agent.algorithm.offline_algorithm_cfg.behavior_cloning_cfg.experts_path=[<abs>/exported/policy.pt]'

# 3) convert to GR00T LeRobot v2.1
python convert_to_lerobot.py --hdf5 demos.hdf5 --out omnireset_peg_insertion \
    --task "insert the peg into the hole" --cams front_rgb side_rgb wrist_rgb --cam-names front side wrist

What the generated dataset looks like

—GR00T LeRobot v2.1: data/*.parquet + videos/observation.images.{front,side,wrist}/*.mp4 + meta/*.json
—action (7): relative Cartesian-OSC end-effector delta [0:6] + binary gripper [6]
—observation.state (7): 6 UR5e joint positions + gripper
—3 cameras @ 224×224 (matches GR00T image_target_size), 10 fps
—Scene: white background, fixed object colors (color-conditioned VLA), fixed lighting
—Reset states: 4 types uniform (peg not-grasped / grasped-on-table / grasped-mid-air / partially-assembled)

Key fixes that make collection work (see `docs/collect_demo.md`)

The stock RGB collection env yields ~0 successful insertions because it (a) uses a 10× too small action scale (z=0.002) and (b) randomizes arm sysid + OSC gains every reset — both break the expert. code/data_collection_rgb_cfg.py fixes these (training action scale + no sysid/gain randomization), taking yield from ~0 to ~20 demos/min with ~85% real pick-and-place insertions.

Fine-tune GR00T

See code/finetune.md. In short: