geonmin-kim/robomimic-can-paired-lerobot
This dataset was created using LeRobot. Dataset Description robomimic can-paired, converted to the LeRobot v3.0 format with success/failure labels. The original robomimic can-paired dataset contains 200 teleoperated demonstrations of the robosuite PickPlaceCan task (Panda robot): 100 paired task initializations, each with one successful demo (the can is picked up and placed in the correct bin) and one failed demo (the can is picked up and tossed outside the robot… See the full description on the dataset page: https://huggingface.co/datasets/geonmin-kim/robomimic-can-paired-lerobot.
This dataset was created using LeRobot.
<a class="flex" href="https://huggingface.co/spaces/lerobot/visualize_dataset?path=geonmin-kim/robomimic-can-paired-lerobot"> <img class="block dark:hidden" src="https://huggingface.co/datasets/huggingface/badges/resolve/main/visualize-this-dataset-xl.svg"/> <img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface/badges/resolve/main/visualize-this-dataset-xl-dark.svg"/> </a>
Dataset Description
robomimic can-paired, converted to the LeRobot v3.0 format with success/failure labels.
The original robomimic can-paired dataset contains 200 teleoperated demonstrations of the robosuite PickPlaceCan task (Panda robot): 100 paired task initializations, each with one successful demo (the can is picked up and placed in the correct bin) and one failed demo (the can is picked up and tossed outside the robot workspace).
Camera observations (agentview, robot0_eye_in_hand wrist camera) were re-rendered offscreen with MuJoCo / robosuite v1.5.1 at 256x256 from the raw demo_v15.hdf5 states.
Success / failure labels
- Per frame: boolean feature
next.success(constant within an episode). - Per episode: the task string ends with
(success)or(failure), so episodes can be filtered via the task metadata. Exactly 100 success / 100 failure episodes.
Labels were derived from the sparse task reward (success = max reward > 0) and cross-checked against the official robomimic low_dim_v15.hdf5 rewards.
Features
observation.images.agentview,observation.images.wrist: 256x256x3 video @ 20 fpsobservation.state(9): eef position (3), eef quaternion (4), gripper qpos (2)observation.environment_state(14): object (can) pose featuresaction(7): OSC delta pose (6) + gripper (1)next.reward,next.done,next.success
Source
- Original data: amandlek/robomimic
v1.5/can/paired - robomimic: Mandlekar et al., "What Matters in Learning from Offline Human Demonstrations for Robot Manipulation", CoRL 2021
- Homepage: https://robomimic.github.io/
- Paper: https://arxiv.org/abs/2108.03298
- License: mit
Zero-shot checkpoint (for the preference-learning assignment)
The designated zero-shot policy for experiments on this dataset is `lerobot/smolvla_base` (~450M, VLM backbone + flow-matching action expert).
- Evaluate it as-is (no additional training) for the Zero-shot row of the comparison table.
smolvla_basewas pretrained mostly on SO-10x real-robot community data, so its zero-shot closed-loop success rate on the robosuitePickPlaceCan(Panda) task is expected to be very low. A low zero-shot number is expected and is not a problem; report the measured value.- Map this dataset's
observation.state(9) /action(7) into SmolVLA's padded state/action convention, and document the mapping in your report. - Official measured zero-shot closed-loop success rate: TBD (will be filled in after in-house measurement).
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "panda",
"total_episodes": 200,
"total_frames": 19795,
"total_tasks": 2,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 20,
"splits": {
"train": "0:200"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path": "videos/{video_key}/chunk-{chunk_index:03d}/file-{file_index:03d}.mp4",
"features": {
"observation.images.agentview": {
"dtype": "video",
"shape": [
256,
256,
3
],
"names": [
"height",
"width",
"channels"
],
"info": {
"video.height": 256,
"video.width": 256,
"video.codec": "av1",
"video.pix_fmt": "yuv420p",
"video.is_depth_map": false,
"video.fps": 20,
"video.channels": 3,
"has_audio": false
}
},
"observation.images.wrist": {
"dtype": "video",
"shape": [
256,
256,
3
],
"names": [
"height",
"width",
"channels"
],
"info": {
"video.height": 256,
"video.width": 256,
"video.codec": "av1",
"video.pix_fmt": "yuv420p",
"video.is_depth_map": false,
"video.fps": 20,
"video.channels": 3,
"has_audio": false
}
},
"observation.state": {
"dtype": "float32",
"shape": [
9
],
"names": [
"eef_pos_x",
"eef_pos_y",
"eef_pos_z",
"eef_quat_x",
"eef_quat_y",
"eef_quat_z",
"eef_quat_w",
"gripper_qpos_0",
"gripper_qpos_1"
]
},
"observation.environment_state": {
"dtype": "float32",
"shape": [
14
],
"names": [
"object_0",
"object_1",
"object_2",
"object_3",
"object_4",
"object_5",
"object_6",
"object_7",
"object_8",
"object_9",
"object_10",
"object_11",
"object_12",
"object_13"
]
},
"action": {
"dtype": "float32",
"shape": [
7
],
"names": [
"dx",
"dy",
"dz",
"droll",
"dpitch",
"dyaw",
"gripper"
]
},
"next.reward": {
"dtype": "float32",
"shape": [
1
],
"names": null
},
"next.done": {
"dtype": "bool",
"shape": [
1
],
"names": null
},
"next.success": {
"dtype": "bool",
"shape": [
1
],
"names": null
},
"timestamp": {
"dtype": "float32",
"shape": [
1
],
"names": null
},
"frame_index": {
"dtype": "int64",
"shape": [
1
],
"names": null
},
"episode_index": {
"dtype": "int64",
"shape": [
1
],
"names": null
},
"index": {
"dtype": "int64",
"shape": [
1
],
"names": null
},
"task_index": {
"dtype": "int64",
"shape": [
1
],
"names": null
}
}
}Citation
BibTeX:
[More Information Needed]