datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
video-to-data-robot-dexterity-task-library-and-dataset
Video to Data: Robot Dexterity Task Library and Dataset
Dataset Description
This dataset contains samples of human demonstrations on manipulation tasks retargeted to bimanual Sharpa robot hands and episodes of robot executions that mimic the original human demonstrations. The former allows a Video to Data user to easily experiment with the Video to Data grounding pipeline, and the latter is an example of the grounded robot data that can be generated with the Video… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/video-to-data-robot-dexterity-task-library-and-dataset.robot-maniskill-image-state-dataset-v1
Robot ManiSkill Image–State Dataset
0. Update
12.22
pickcube_rgb_random_v2: having a broader distribution of viewpoints and qpos values
12.14
pickcube_rgb_traj_v3: official PickCube-v1 scene RGBD trajectories (Type-1 scaling baseline)
12.2
delete pickcube_rgb_traj_v1
11.30
pickcube_rgb_traj_v2: rendering images with cube in pickcube task.
11.28
pickcube_rgb_traj_v1: Trajectory subset… See the full description on the dataset page: https://huggingface.co/datasets/surgingTu/robot-maniskill-image-state-dataset-v1.touch_in_the_wild-dataset
📦 Touch in the Wild
Learning Fine-Grained Manipulation with a Portable Visuo-Tactile Gripper
🔍 Overview
This dataset supports the paper Touch in the Wild: Learning Fine-Grained Manipulation with a Portable Visuo-Tactile Gripper. It contains a wide range of visuo-tactile demonstrations collected using a portable visuo-tactile gripper, spanning both structured lab settings and unstructured real-world environments.
The dataset is divided into three major categories:
Four… See the full description on the dataset page: https://huggingface.co/datasets/binghaohuang-robot/touch_in_the_wild-dataset.KIT_IRL_Real_Robot_Kitchen_Dataset_Vis_OnlyDataset with joint state play data for a toy kitchen. Does not contain language annotations.
The dataset contains 20 different tasks with two static cameras.
@inproceedings{
reuss2024multimodal,
title={Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goals},
author={Moritz Reuss and {\"O}mer Erdin{\c{c}} Ya{\u{g}}murlu and Fabian Wenzel and Rudolf Lioutikov},
booktitle={Robotics: Science and Systems},
year={2024}
}
arx5-robot-dataset
ARX5 Manipulation Dataset
Robotic manipulation demos collected on a single ARX5 arm, of tabletop tasks with dual RGB cameras and DINOv2 embeddings of the images.
Converted from Ville Kuosmanen's LeRobot v2.1 datasets into a unified HDF5 format.
Hardware
Robot: ARX5 single arm (7-DoF)
Cameras: Wrist-mounted + front-facing scene camera
Stats
Trajectories
1,704
Total frames
860,076
Frames per trajectory
4 – 3,034 (median 397)
Total size
1 TB+… See the full description on the dataset page: https://huggingface.co/datasets/pravsels/arx5-robot-dataset.KIT_IRL_Real_Robot_Kitchen_Dataset_LangDataset with joint state play data for a toy kitchen.
The dataset contains 20 different tasks with two static cameras.
@inproceedings{
reuss2024multimodal,
title={Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goals},
author={Moritz Reuss and {\"O}mer Erdin{\c{c}} Ya{\u{g}}murlu and Fabian Wenzel and Rudolf Lioutikov},
booktitle={Robotics: Science and Systems},
year={2024}
}
diverse-robot-datasetrobot-action-prediction-dataset
Robotic Action Prediction Dataset
Dataset Description
This dataset contains triplets of (current observation, action instruction, future observation) for training models to predict future frames of robotic actions.
Dataset Structure
Data Fields
current_frame: Input image (RGB) of the current observation
instruction: Textual description of the action to perform
future_frame: Target image (RGB) showing the expected outcome 50 frames later… See the full description on the dataset page: https://huggingface.co/datasets/bryandts/robot-action-prediction-dataset.robot_jersey_dataset
NUpbr RoboCup Jersey-Colour Robot Detection Dataset
1,707 physically-based-rendered images of RoboCup Humanoid Soccer scenes,
generated with NUpbr, each
containing 1-4 robots and one ball, with per-instance bounding boxes and,
for each robot, a jersey colour.
Every frame is restricted to at most two distinct jersey colours across all
robots present, regardless of robot count. Each robot is independently
assigned one of the two colours sampled fresh for that frame from a wide… See the full description on the dataset page: https://huggingface.co/datasets/Ysobel/robot_jersey_dataset.robot_pickup_datasetThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "so101_follower",
"total_episodes": 50,
"total_frames": 10207,
"total_tasks": 1,
"total_videos": 0,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 30,
"splits": {
"train": "0:50"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path": null,
"features": {… See the full description on the dataset page: https://huggingface.co/datasets/DozenDucc/robot_pickup_dataset.calibra-robot-dataset-quality-benchmark
Calibra Robot Dataset Quality Benchmark
Automated quality audits of 30 public LeRobot
datasets, produced by Calibra.
Mean health score: 85.8 / 100 across 30 datasets (206–800 episodes each).
Key finding: every audited dataset has at least one critical quality flag,
confirming that silent data issues are the norm, not the exception, in public
robot demonstration corpora.
Calibra Community Dataset Quality Benchmark
Generated 2026-07-27 19:07 UTC by Calibra
Audited… See the full description on the dataset page: https://huggingface.co/datasets/omert27/calibra-robot-dataset-quality-benchmark.chris_robot_dataset_with_tracesThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "panda",
"total_episodes": 4,
"total_frames": 1272,
"total_tasks": 1,
"total_videos": 0,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 15,
"splits": {
"train": "0:4"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/brandonyang/chris_robot_dataset_with_traces.robot-bottle-datasetThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "panda",
"total_episodes": 331,
"total_frames": 140760,
"total_tasks": 2,
"total_videos": 0,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 15,
"splits": {
"train": "0:331"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/jhalwaysthespecial/robot-bottle-dataset.combined_chris_robot_datasetThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "panda",
"total_episodes": 0,
"total_frames": 0,
"total_tasks": 0,
"total_videos": 0,
"total_chunks": 0,
"chunks_size": 1000,
"fps": 15,
"splits": {},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/brandonyang/combined_chris_robot_dataset.video-to-data-robot-dexterity-task-library-and-dataset
Video to Data: Robot Dexterity Task Library and Dataset
Dataset Description
This dataset contains samples of human demonstrations on manipulation tasks retargeted to bimanual Sharpa robot hands and episodes of robot executions that mimic the original human demonstrations. The former allows a Video to Data user to easily experiment with the Video to Data grounding pipeline, and the latter is an example of the grounded robot data that can be generated with the Video… See the full description on the dataset page: https://huggingface.co/datasets/introvoyz042/video-to-data-robot-dexterity-task-library-and-dataset.robot-bottle-datasetThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "panda",
"total_episodes": 331,
"total_frames": 140760,
"total_tasks": 2,
"chunks_size": 1000,
"fps": 15,
"splits": {
"train": "0:331"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path": "videos/{video_key}/chunk-{chunk_index:03d}/file-{file_index:03d}.mp4",
"features": {… See the full description on the dataset page: https://huggingface.co/datasets/michios/robot-bottle-dataset.IB_Robot_dataset_banana_pick
数据集：1arm_2cam_banana_pick_v1_20260514
创建日期：2026-05-14
任务类型：banana_pick
任务描述：1臂2相机抓取香蕉放到盘子里
数据版本：v1
数据来源：实机采集（正常轨迹录制）
关联训练中间模型：暂无
关联可用模型：暂无
数据集详情
机器人类型：so_follower
相机数量：2（top, wrist）
总 episode 数：80
总帧数：45358
FPS：30
图像分辨率：480x640
视频编码：av1
动作维度：6（shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper）
Episode 分布
所有数据均为正常轨迹录制，香蕉初始位置按以下方式分布在盘子周围：
一、香蕉横放（episode 0~39）
Episode 范围
香蕉初始位置
0 ~ 9
盘子左侧
10 ~ 19… See the full description on the dataset page: https://huggingface.co/datasets/openEuler/IB_Robot_dataset_banana_pick.chris_robot_dataset_with_traces_fixedThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "panda",
"total_episodes": 87,
"total_frames": 19814,
"total_tasks": 6,
"total_videos": 0,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 15,
"splits": {
"train": "0:87"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/brandonyang/chris_robot_dataset_with_traces_fixed.robot-trajectories-datasetrobot-kinematics-dataset
🤖 EgoCentric Multi-Episode Kinematics Trajectory Dataset
This repository contains multi-episode egocentric (first-person) video recordings paired with extracted 3D hand, wrist, and arm kinematics trajectories. The dataset is specifically structured for Imitation Learning (IL), Behavior Cloning (BC), Isaac Sim Kinematics Replay, and Reinforcement Learning Datasets (RLDS) training pipelines.
📌 Dataset Overview
Source Domain
Egocentric /… See the full description on the dataset page: https://huggingface.co/datasets/shiyixia/robot-kinematics-dataset.robot-intelligence-dataset
🤖 Robot Intelligence Dataset
A collection of three real-world robotics sensor datasets used to train and evaluate machine learning pipelines across four intelligence challenges:
Perception · Navigation · Failure Detection · Autonomous Decision-Making (RL).
Companion GitHub repo: KushSaraf/Robot_Intelligence
Made by: Kush Saraf & Yash Chavda
Datasets
🧠 1. Perception Dataset — Human Activity Recognition (HAR)
Source: UCI HAR Dataset
Property… See the full description on the dataset page: https://huggingface.co/datasets/kushsaraf/robot-intelligence-dataset.dual-arm-robot-dataset
Dual-Arm Robot Dataset
A LeRobot-compatible dataset containing joint position data from a dual-arm robotic system with 34 degrees of freedom.
Dataset Description
This dataset contains robot manipulation data recorded from a dual-arm robotic system. The data includes joint positions for both arms and their associated grippers, captured at 100Hz for precise motion analysis and learning.
Dataset Summary
Total Episodes: 1
Total Frames: 3,105
Duration: 31.05… See the full description on the dataset page: https://huggingface.co/datasets/Sraghvi/dual-arm-robot-dataset.my-first-robot-datasetThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "so101_follower",
"total_episodes": 5,
"total_frames": 7567,
"total_tasks": 1,
"total_videos": 5,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 30,
"splits": {
"train": "0:5"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/erebius/my-first-robot-dataset.le-robot-merged-datasetIB_Robot_dataset_dual_arm_banana_pick
数据集：2arm_3cam_banana_pick_v1_20260617
创建日期：2026-06-17
任务类型：banana_pick
任务描述：双臂3相机抓取香蕉放到盒子（左臂+右臂协同操作）
数据版本：v1
数据来源：实机采集（正常轨迹录制）
关联训练中间模型：暂无
关联可用模型：暂无
数据集详情
机器人类型：bi_so100_follower
相机数量：3（top, left, right）
总 episode 数：139
总帧数：87699
FPS：30
图像分辨率：240x320
视频编码：av1
动作维度：12（left: shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper; right: shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper）
Episode 分布
Episode… See the full description on the dataset page: https://huggingface.co/datasets/openEuler/IB_Robot_dataset_dual_arm_banana_pick.robot_datasetThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/ohmyhong/robot_dataset.Dataset_Robot_IA_full_v8Arfan-VLA-koch-robot-datasetThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "koch_follower",
"total_episodes": 11,
"total_frames": 7904,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:11"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/Groot2021/Arfan-VLA-koch-robot-dataset.safe_rl_point_robot_datasetshrugmaster-robot-dataset
Shrugmaster_Bimanual_Manipulation
Robot manipulation dataset with 2 episodes.
Dataset Structure
Episodes: 2 episodes of robot manipulation
Cameras: 3 camera views (base, arm1, arm2)
Joints: 34 joint states
Format: LeRobot v2.1
Usage
from lerobot.common.datasets import LeRobotDataset
dataset = LeRobotDataset("Sraghvi/shrugmaster-robot-dataset")
Files
data/chunk-000/: Parquet files with episode data
videos/chunk-000/: MP4 video files for each… See the full description on the dataset page: https://huggingface.co/datasets/Sraghvi/shrugmaster-robot-dataset.