datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
robot-maniskill-image-state-dataset-v1
Robot ManiSkill Image–State Dataset
0. Update
12.22
pickcube_rgb_random_v2: having a broader distribution of viewpoints and qpos values
12.14
pickcube_rgb_traj_v3: official PickCube-v1 scene RGBD trajectories (Type-1 scaling baseline)
12.2
delete pickcube_rgb_traj_v1
11.30
pickcube_rgb_traj_v2: rendering images with cube in pickcube task.
11.28
pickcube_rgb_traj_v1: Trajectory subset… See the full description on the dataset page: https://huggingface.co/datasets/surgingTu/robot-maniskill-image-state-dataset-v1.ManiSkill_Demonstrations
ManiSkill Demonstrations
This dataset repo contains all of the latest ManiSkill demonstration datasets as well as some pretained model weights used to generate some demonstrations. To download by environment ID run
python -m mani_skill.utils.download_demo env_id
You can also replace env_id with "all" to download all demos.
For the previous ManiSkill2 paper's demonstration data (v0.5.3 of the mani_skill package) see https://huggingface.co/datasets/haosulab/ManiSkill2
The… See the full description on the dataset page: https://huggingface.co/datasets/haosulab/ManiSkill_Demonstrations.ManiSkill2
ManiSkill2 Data
Update: ManiSkill 3 has been released https://github.com/haosulab/ManiSkill/. It uses different datasets than ManiSkill2 so the data here is not expected to transfer over
ManiSkill2 is a unified benchmark for learning generalizable robotic manipulation skills powered by SAPIEN. It features 20 out-of-box task families with 2000+ diverse object models and 4M+ demonstration frames. Moreover, it empowers fast visual input learning algorithms so that a CNN-based… See the full description on the dataset page: https://huggingface.co/datasets/haosulab/ManiSkill2.RPD-maniskill
Maniskill Sub-Dataset in RLDS Format used in RPD
This repository contains the maniskill subset in RLDS format used to train Octo and OpenVLA in the paper Refined Policy Distillation: From VLA Generalists to RL Experts, which distilled these models using RL.
Project page: https://refined-policy-distillation.github.io
Code: https://github.com/Refined-Policy-Distillation/RPD
Checkout octo-base-1.5-finetuned-maniskill and openvla-7b-finetuned-maniskill which have been fine-tuned with… See the full description on the dataset page: https://huggingface.co/datasets/Juelg/RPD-maniskill.rlt-maniskill-PegInsertionSide-v1-400-succ
RLT ManiSkill Joint
Dataset Summary
rlt_maniskill_joint is a LeRobot-style dataset for joint-control Robot Learning Token (RLT) training on the ManiSkill peg insertion task.
It is designed for the RLinf + OpenPI pi05_rlt_joint pipeline and is used in three stages:
OpenPI supervised fine-tuning (SFT) base policy training
RLT Stage 1 RL-token training
RLT Stage 2 online RL initialization and normalization
The dataset corresponds to the ManiSkill task:… See the full description on the dataset page: https://huggingface.co/datasets/RLinf/rlt-maniskill-PegInsertionSide-v1-400-succ.maniskill_dataset_rawManiSkill_bridge_v2_real2simAssets for Real2Sim evaluation of the Bridge v2 dataset from https://github.com/simpler-env/SimplerEnv/
maniskill_allmaniskill_assets
Asset Download
The assets need to be placed into RLinf's ManiSkill environment folder with the name assets.
# uv pip install huggingface_hub if you don't have it
cd <path_to_RLinf>/rlinf/envs/maniskill
hf download --repo-type dataset RLinf/maniskill_assets --local-dir ./assets
You can also use git to clone the repository:
cd <path_to_RLinf>/rlinf/envs/maniskill
git clone https://hf.co/datasets/RLinf/maniskill_assets ./assets
License
Our assets are attributed to… See the full description on the dataset page: https://huggingface.co/datasets/RLinf/maniskill_assets.ManiSkill_manipulatemaniskill-dreamer4-expertmaniskill3-sft-rgb-lerobot-1200epThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "maniskill3_sim",
"total_episodes": 1200,
"total_frames": 171480,
"total_tasks": 6,
"total_videos": 0,
"total_chunks": 2,
"chunks_size": 1000,
"fps": 20,
"splits": {
"train": "0:1200"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path": null,
"features":… See the full description on the dataset page: https://huggingface.co/datasets/onnoboru/maniskill3-sft-rgb-lerobot-1200ep.maniskill-cubeinbowl-lerobotconflict_experiment_maniskill
ManiSkill 迁移指南（迁移到另一台工作站）
本文档说明如何将 datagen/ManiSkill 下的代码与工作流迁移到另一台工作站环境，并在新机器上完成依赖安装与数据生成流程。
1. 项目简介与目录结构
本仓库中的 ManiSkill 用于数据生成：通过 motion planning 生成 Pick/Pull 等任务的轨迹，replay 成带 RGB 的 H5，再转换为 LeRobot 格式数据集。
主要目录与脚本：
路径
说明
mani_skill/
ManiSkill 核心包（envs、agents、trajectory 等）
scripts/
数据生成脚本（如 run_verb_color_shape_motion_planning.py）
demo_collection/
收集 demo + 转 LeRobot 的脚本与依赖
demos/
生成的轨迹 H5 与 JSON（可迁移或在新环境重新生成）
lerobot_dataset/
转换后的 LeRobot 数据集输出目录… See the full description on the dataset page: https://huggingface.co/datasets/yqi19/conflict_experiment_maniskill.maniskill_50ep_so101_blue_cube_orange_tray_20260812_131142This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/makermods/maniskill_50ep_so101_blue_cube_orange_tray_20260812_131142.maniskill_50ep_so101_blue_cube_orange_tray_20260811_114951This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/makermods/maniskill_50ep_so101_blue_cube_orange_tray_20260811_114951.ManiSkill_TwoRobotStackCube-v1_recovery
TwoRobotStackCube-v1 Dataset
This dataset was converted from ManiSkill format to LeRobot format. It is part of the paper CroSTAta: Cross-State Transition Attention Transformer for Robotic Manipulation.
Code: GitHub Repository
Authors: Giovanni Minelli, Giulio Turrisi, Victor Barasuol, Claudio Semini
Dataset Information
Environment: TwoRobotStackCube-v1
Total Episodes: 1043
Total Frames: 25763
FPS: 20
Video Keys: ['observation.image']
Target Control Mode: Original… See the full description on the dataset page: https://huggingface.co/datasets/johnMinelli/ManiSkill_TwoRobotStackCube-v1_recovery.Maniskill-Pushcube-demonstration-largeThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.0",
"robot_type": null,
"total_episodes": 1860,
"total_frames": 0,
"total_tasks": 0,
"total_videos": 0,
"total_chunks": 0,
"chunks_size": 1000,
"fps": 20,
"splits": {},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/AdilZtn/Maniskill-Pushcube-demonstration-large.maniskill_pi0This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.0",
"robot_type": "panda",
"total_episodes": 200,
"total_frames": 21321,
"total_tasks": 1,
"total_videos": 0,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 50,
"splits": {
"train": "0:200"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/Ruoxiang/maniskill_pi0.ManiSkill3-zhiyuan-8tasksManiskill-Pushcube-demonstration-mediumThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.0",
"robot_type": null,
"total_episodes": 500,
"total_frames": 0,
"total_tasks": 0,
"total_videos": 0,
"total_chunks": 0,
"chunks_size": 1000,
"fps": 20,
"splits": {},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/AdilZtn/Maniskill-Pushcube-demonstration-medium.real_world_maniskill_lift_peg_eefManiSkill
ManiSkill Data
ManiSkill is a unified benchmark for learning generalizable robotic manipulation skills powered by SAPIEN. It features 20 out-of-box task families with 2000+ diverse object models and 4M+ demonstration frames. Moreover, it empowers fast visual input learning algorithms so that a CNN-based policy can collect samples at about 2000 FPS with 1 GPU and 16 processes on a workstation. The benchmark can be used to study a wide range of algorithms: 2D & 3D vision-based… See the full description on the dataset page: https://huggingface.co/datasets/haosulab/ManiSkill.ManiSkill_StackCube-v1_recovery
StackCube-v1 Dataset
This dataset was converted from ManiSkill format to LeRobot format for the paper CroSTAta: Cross-State Transition Attention Transformer for Robotic Manipulation.
Code: https://github.com/iit-DLSLab/croSTAta
Checkpoints: CroSTAta Evaluation Checkpoints
Dataset Information
Environment: StackCube-v1
Total Episodes: 1010
Total Frames: 34249
FPS: 20
Video Keys: ['observation.images.base_camera', 'observation.images.hand_camera']
Target Control Mode:… See the full description on the dataset page: https://huggingface.co/datasets/johnMinelli/ManiSkill_StackCube-v1_recovery.ManiSkill-1000ManiSkill_PickCube
ManiSkill PickCube Demonstrations
Contains the following demonstrations:
Motion Planning demonstrations
Human (via interactive motion planning) demonstrations
Reinforcement Learning demonstrations
maniskill-dataManiSkill_PegInsertionSide
ManiSkill PegInsertionSide Demonstrations
Contains demonstrations sourced through the following methods:
Motion Planning (~1000)
Reinforcement Learning demonstrations (~1000)
real_world_maniskill_lift_pegmaniskill3