datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
PhysicalAI-Robotics-Manipulation-Kitchen
PhysicalAI Robotics Manipulation in the Kitchen
Dataset Description:
PhysicalAI-Robotics-Manipulation-Kitchen is a dataset of automatic generated motions of robots performing operations such as opening and closing cabinets, drawers, dishwashers and fridges. The dataset was generated in IsaacSim leveraging reasoning algorithms and optimization-based motion planning to find solutions to the tasks automatically [1, 3]. The dataset includes a bimanual manipulator built with… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-Manipulation-Kitchen.Arena-G1-Loco-Manipulation-Task
Dataset Description:
The Arena-G1-Loco-Manipulation-Task dataset is multimodal collections of trajectories generated in Isaac Lab. It supports humanoid (G1) loco-manipulation task in IsaacLab-Arena environment. Each entry provides the full context (state, vision, language, action) needed to train and evaluate generalist robot policies for box pick and place task.
Dataset Name
# Trajectories
G1 Loco-Manipulation Task
50
This dataset is ideal for behavior cloning… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Arena-G1-Loco-Manipulation-Task.berkeley_fanuc_manipulation_lerobotThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.0",
"robot_type": "fanuc_mate",
"total_episodes": 415,
"total_frames": 62613,
"total_tasks": 32,
"total_videos": 830,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 10,
"splits": {
"train": "0:415"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/IPEC-COMMUNITY/berkeley_fanuc_manipulation_lerobot.PhysicalAI-Robotics-Manipulation-ObjectsPhysicalAI-Robotics-Manipulation-Objects is a dataset of automatic generated motions of robots performing operations such as picking and placing objects in a kitchen environment. The dataset was generated in IsaacSim leveraging reasoning algorithms and optimization-based motion planning to find solutions to the tasks automatically [1, 3]. The dataset includes a bimanual manipulator built with Kinova Gen3 arms. The environments are kitchen scenes where the furniture and appliances were… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-Manipulation-Objects.utokyo_pr2_tabletop_manipulationThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.0",
"robot_type": "unknown",
"total_episodes": 240,
"total_frames": 32708,
"total_tasks": 3,
"total_videos": 240,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 5,
"splits": {
"train": "0:240"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/lerobot/utokyo_pr2_tabletop_manipulation.factory-manipulation-videos
Factory manipulation videos
Procedural Robotics is open sourcing a small set of our factory data so teams can assess its quality. The videos show workers performing factory tasks.
Contents
Seven continuous takes, 109 minutes in total.
Task
Station
Worker
Duration
File
cardboard manipulation
01
041
23.6 min
cardboard_manipulation_station01_worker041.mp4
cardboard manipulation
04
026
16.5 min
cardboard_manipulation_station04_worker026.mp4
defect… See the full description on the dataset page: https://huggingface.co/datasets/procedural-robotics/factory-manipulation-videos.yam_bimanual_manipulation
YAM Bimanual Manipulation
Real-robot demonstrations for three bimanual manipulation tasks, collected on a
YAM dual-arm platform and stored in
LeRobot v3.0 format.
Each task is a self-contained LeRobot dataset in its own top-level directory:
Directory
Task
Episodes
Frames
Size
blocks_filtered/
Put all blocks into the box.
97
136,489
2.3 GB
dustpan_filtered/
Clean the table using the dust pan.
100
58,018
773 MB
transfer_filtered/
Transfer the egg from the pan into… See the full description on the dataset page: https://huggingface.co/datasets/shailes-h/yam_bimanual_manipulation.berkeley_fanuc_manipulationThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.0",
"robot_type": "unknown",
"total_episodes": 415,
"total_frames": 62613,
"total_tasks": 32,
"total_videos": 830,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 10,
"splits": {
"train": "0:415"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/lerobot/berkeley_fanuc_manipulation.Arena-GR1-Manipulation-PlaceItemCloseDoor-Task
Dataset Description:
The Arena-GR1-Manipulation-PlaceItemCloseDoor-Task dataset is a multimodal collection of trajectories generated in Isaac Lab. It supports humanoid (GR1) manipulation tasks in the IsaacLab-Arena environment. Each entry provides the full context (state, vision, language, and action) needed to train and evaluate generalist robot policies for a sequential task (e.g. putting object into a fridge and closing the door).
Dataset Name
# Trajectories
GR1… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Arena-GR1-Manipulation-PlaceItemCloseDoor-Task.Arena-GR1-Manipulation-Task
Dataset Description:
The Arena-GR1-Manipulation-Task dataset is multimodal collections of trajectories generated in Isaac Lab. It supports humanoid (GR1) manipulation task in IsaacLab-Arena environment. Each entry provides the full context (state, vision, language, action) needed to train and evaluate generalist robot policies for opening microwave task.
Dataset Name
# Trajectories
GR1 Manipulation Task
50
This dataset is ideal for behavior cloning, policy learning… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Arena-GR1-Manipulation-Task.conq_hose_manipulationThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "unknown",
"total_episodes": 139,
"total_frames": 8277,
"total_tasks": 3,
"chunks_size": 1000,
"fps": 30,
"splits": {
"train": "0:139"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path": "videos/{video_key}/chunk-{chunk_index:03d}/file-{file_index:03d}.mp4"… See the full description on the dataset page: https://huggingface.co/datasets/lerobot/conq_hose_manipulation.zerograv-manipulation-trajectories-v0
⚠️ Superseded — do not use for distillation
These trajectories were collected from policies trained before the sim-real
action-space alignment. Measured on this data: mean |arm action| is 0.72, with
46-53% of actions at the clip boundary of [-1, 1]. For comparison, the real
robot's own pi0.5-DROID policy outputs a mean of 0.066 and human teleop 0.101
— so these actions sit 7-11x outside the distribution the base model has seen.
Distilling them would push pi0.5-DROID's action… See the full description on the dataset page: https://huggingface.co/datasets/stray-light/zerograv-manipulation-trajectories-v0.lerobot-so-101-manipulations
So-101 Dataset
The dataset comprises 20 episodes of high-quality teleoperated robotic arm recordings at ~30 FPS. It is a compact, curated set captured using multi-view cameras (front, wrist, top) within the calibrated robot studio of the SO-101 arm, leveraging the LeRobot framework. Specifically designed for imitation learning and model training on the LeRobot platform, this dataset demonstrates basic manipulation and pick-and-place tasks with structured LeRobot v2.1… See the full description on the dataset page: https://huggingface.co/datasets/UniDataPro/lerobot-so-101-manipulations.crab-manipulation-dataset-labeled
Crab Manipulation Dataset — Reward Labeled
Same demonstrations as crab-manipulation-dataset, augmented with per-episode reward parameters for Safety-Aware Reward-Weighted Flow Matching (SA-RWFM) training.
What's Different
Each session's meta/ folder contains an additional reward_params.json with:
Tactile analysis: contact fractions, force statistics, damage flags
Gripper analysis: open/close thresholds, velocity analysis
Reward parameters: success/drop/damage rewards… See the full description on the dataset page: https://huggingface.co/datasets/armteam/crab-manipulation-dataset-labeled.so101_gba_direct_manipulationThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "so101_leader",
"total_episodes": 200,
"total_frames": 47567,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:200"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/giacomoran/so101_gba_direct_manipulation.factory-manipulation-lerobot-v2egocentric-manipulation-sample
SmartDeer — Egocentric Human Demonstration Data (Public Sample)
SmartDeer collects egocentric human demonstration data for embodied AI: real operators, real tasks, real environments. No teleoperation rigs, no lab mock-ups, no actors.
This repository is a small public sample, published so you can inspect our capture quality, annotation schema and file conventions. Production data is delivered under commercial terms.
This sample
Task twist_cap — bimanual bottle-cap… See the full description on the dataset page: https://huggingface.co/datasets/SmartDeer/egocentric-manipulation-sample.lerobot-so-101-manipulations
So-101 Dataset
The dataset comprises 20 episodes of high-quality teleoperated robotic arm recordings at ~30 FPS. It is a compact, curated set captured using multi-view cameras (front, wrist, top) within the calibrated robot studio of the SO-101 arm, leveraging the LeRobot framework. Specifically designed for imitation learning and model training on the LeRobot platform, this dataset demonstrates basic manipulation and pick-and-place tasks with structured LeRobot v2.1 annotations.-… See the full description on the dataset page: https://huggingface.co/datasets/ud-smart-city/lerobot-so-101-manipulations.Chinese_Commercial_Kitchen_Manipulation_Dataset_Preview
🍳 Chinese Commercial Kitchen Manipulation Dataset — Sample Pack v0.1
Asia's first real commercial kitchen manipulation dataset.Professional chef (20 years) · Real restaurant environment · Multi-view RGB-D · Egocentric video
📧 Request evaluation samples or full data: andy@dynamicnova.com
Overview
This sample pack contains real-world cooking demonstrations collected in an operating Chinese commercial kitchen in Zhongshan, Guangdong, China.
The data focuses on… See the full description on the dataset page: https://huggingface.co/datasets/nova-dynamics/Chinese_Commercial_Kitchen_Manipulation_Dataset_Preview.Arena-GR1-Manipulation-Task-v3This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "GR1",
"total_episodes": 50,
"total_frames": 4928,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 50,
"splits": {
"train": "0:50"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Arena-GR1-Manipulation-Task-v3.oversonic-r1916-manipulation-v4This dataset was created using LeRobot v3.0 format via Forge.
Robot: oversonic (21-DOF)
Cameras: observation.images.nav, observation.images.head, observation.images.wrist_left
Episodes: 2
Frames: 378
FPS: 27
crab-manipulation-dataset
Crab Manipulation Dataset
Teleoperated demonstration dataset for contact-rich manipulation on the Crab bimanual mobile manipulator (dual SO-101 arms). Collected at 15 Hz with 3 RGB cameras (640×480) and dual 10×10 tactile force sensor matrices.
Dataset Structure
Clean demonstrations (27 sessions, used for training)
Folder
Sessions
Task
Hardness
egg_carton_to_tray/
11
Open carton, place egg on tray
Soft
marmalade_jar/
7
Pick and place jar
Medium… See the full description on the dataset page: https://huggingface.co/datasets/armteam/crab-manipulation-dataset.unitree-g1-manipulationThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "Unitree_G1_Dex3",
"total_episodes": 2,
"total_frames": 603,
"total_tasks": 1,
"chunks_size": 1000,
"fps": 30,
"splits": {
"train": "0:2"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path": "videos/{video_key}/chunk-{chunk_index:03d}/file-{file_index:03d}.mp4"… See the full description on the dataset page: https://huggingface.co/datasets/Breno-de-Angelo/unitree-g1-manipulation.GR1-Manipulation-IsaacSim-Rendered
GR1 Manipulation — Isaac Sim Rendered Dataset
Dataset Description
Multi-modal rendered dataset of NVIDIA GR1 humanoid robot performing tabletop manipulation tasks
in Isaac Sim. Each episode is rendered as 3 synchronized MP4 videos:
RGB — ray-traced color video
Depth — normalized depth map (grayscale, 0–10m range)
Segmentation — semantic segmentation with per-object class coloring
This dataset is designed as input for NVIDIA Cosmos Transfer 2.5 to generate photorealistic… See the full description on the dataset page: https://huggingface.co/datasets/khang123452/GR1-Manipulation-IsaacSim-Rendered.berkeley_fanuc_manipulation_lerobotThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.0",
"robot_type": "fanuc_mate",
"total_episodes": 415,
"total_frames": 62613,
"total_tasks": 32,
"total_videos": 830,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 10,
"splits": {
"train": "0:415"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/introvoyz041/berkeley_fanuc_manipulation_lerobot.ego4d-manipulation-v1so101_cube_direct_manipulationThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "so101_leader",
"total_episodes": 100,
"total_frames": 25582,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:100"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/giacomoran/so101_cube_direct_manipulation.conditional-manipulation-task-6This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "so_follower",
"total_episodes": 53,
"total_frames": 36221,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:53"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/ArshiaE/conditional-manipulation-task-6.reactor-x2-GR1-Manipulation-Task-v3
reactor-x2-GR1-Manipulation-Task-v3
31 episodes, 2,656 frames of GR1 humanoid data, each episode built by adding a
different food item onto the plate/turntable inside the microwave in one real
recorded nvidia/Arena-GR1-Manipulation-Task-v3
episode, using Reactor XMAX X2 video editing. Standard
LeRobot v2.1 layout (meta/,
data/, videos/ at the repo root), so it loads directly with:
from lerobot.common.datasets.lerobot_dataset import LeRobotDataset
ds =… See the full description on the dataset page: https://huggingface.co/datasets/kabilanKB/reactor-x2-GR1-Manipulation-Task-v3.piper-all-manipulation-cleaned-v1-filtered