datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
video-to-data-robot-dexterity-task-library-and-dataset
Video to Data: Robot Dexterity Task Library and Dataset
Dataset Description
This dataset contains samples of human demonstrations on manipulation tasks retargeted to bimanual Sharpa robot hands and episodes of robot executions that mimic the original human demonstrations. The former allows a Video to Data user to easily experiment with the Video to Data grounding pipeline, and the latter is an example of the grounded robot data that can be generated with the Video… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/video-to-data-robot-dexterity-task-library-and-dataset.robot-maniskill-image-state-dataset-v1
Robot ManiSkill Image–State Dataset
0. Update
12.22
pickcube_rgb_random_v2: having a broader distribution of viewpoints and qpos values
12.14
pickcube_rgb_traj_v3: official PickCube-v1 scene RGBD trajectories (Type-1 scaling baseline)
12.2
delete pickcube_rgb_traj_v1
11.30
pickcube_rgb_traj_v2: rendering images with cube in pickcube task.
11.28
pickcube_rgb_traj_v1: Trajectory subset… See the full description on the dataset page: https://huggingface.co/datasets/surgingTu/robot-maniskill-image-state-dataset-v1.data
Dataset Card for BEHAVIOR Robot Suite (BRS) Data
This dataset provides robotic trajectories for five real-world household tasks. These tasks are:
Clean house after a wild party;
Clean the toilet;
Take trash outside;
Put items onto shelves;
Lay clothes out.
These data are first collected and used in the paper BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activities
.
Dataset Details
Dataset Sources… See the full description on the dataset page: https://huggingface.co/datasets/behavior-robot-suite/data.real_robot_data
Real Robot Manipulation Dataset
Tabletop manipulation tasks recorded on a UR5e with a Robotiq 2F-140
gripper. Most tasks use two Intel RealSense cameras (fixed base + wrist-mounted);
the newest hide_block_with_cup task adds a third (opposite-side counter view).
All recordings at 20 Hz.
Tasks
Folder
Eps
Cameras
Instruction
put_blocks_in_plate
100
2
Pick two blocks from the table and place them on the plate.
remove_blocks_from_plate
100
2
Pick two… See the full description on the dataset page: https://huggingface.co/datasets/Hoshipu/real_robot_data.touch_in_the_wild-dataset
📦 Touch in the Wild
Learning Fine-Grained Manipulation with a Portable Visuo-Tactile Gripper
🔍 Overview
This dataset supports the paper Touch in the Wild: Learning Fine-Grained Manipulation with a Portable Visuo-Tactile Gripper. It contains a wide range of visuo-tactile demonstrations collected using a portable visuo-tactile gripper, spanning both structured lab settings and unstructured real-world environments.
The dataset is divided into three major categories:
Four… See the full description on the dataset page: https://huggingface.co/datasets/binghaohuang-robot/touch_in_the_wild-dataset.KIT_IRL_Real_Robot_Kitchen_Dataset_Vis_OnlyDataset with joint state play data for a toy kitchen. Does not contain language annotations.
The dataset contains 20 different tasks with two static cameras.
@inproceedings{
reuss2024multimodal,
title={Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goals},
author={Moritz Reuss and {\"O}mer Erdin{\c{c}} Ya{\u{g}}murlu and Fabian Wenzel and Rudolf Lioutikov},
booktitle={Robotics: Science and Systems},
year={2024}
}
robot_data_for_openvlaKIT_IRL_Real_Robot_Kitchen_Dataset_LangDataset with joint state play data for a toy kitchen.
The dataset contains 20 different tasks with two static cameras.
@inproceedings{
reuss2024multimodal,
title={Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goals},
author={Moritz Reuss and {\"O}mer Erdin{\c{c}} Ya{\u{g}}murlu and Fabian Wenzel and Rudolf Lioutikov},
booktitle={Robotics: Science and Systems},
year={2024}
}
robot-action-prediction-dataset
Robotic Action Prediction Dataset
Dataset Description
This dataset contains triplets of (current observation, action instruction, future observation) for training models to predict future frames of robotic actions.
Dataset Structure
Data Fields
current_frame: Input image (RGB) of the current observation
instruction: Textual description of the action to perform
future_frame: Target image (RGB) showing the expected outcome 50 frames later… See the full description on the dataset page: https://huggingface.co/datasets/bryandts/robot-action-prediction-dataset.diverse-robot-datasetvideo-to-data-robot-dexterity-task-library-and-dataset
Video to Data: Robot Dexterity Task Library and Dataset
Dataset Description
This dataset contains samples of human demonstrations on manipulation tasks retargeted to bimanual Sharpa robot hands and episodes of robot executions that mimic the original human demonstrations. The former allows a Video to Data user to easily experiment with the Video to Data grounding pipeline, and the latter is an example of the grounded robot data that can be generated with the Video… See the full description on the dataset page: https://huggingface.co/datasets/introvoyz042/video-to-data-robot-dexterity-task-library-and-dataset.arx5-robot-dataset
ARX5 Manipulation Dataset
Robotic manipulation demos collected on a single ARX5 arm, of tabletop tasks with dual RGB cameras and DINOv2 embeddings of the images.
Converted from Ville Kuosmanen's LeRobot v2.1 datasets into a unified HDF5 format.
Hardware
Robot: ARX5 single arm (7-DoF)
Cameras: Wrist-mounted + front-facing scene camera
Stats
Trajectories
1,704
Total frames
860,076
Frames per trajectory
4 – 3,034 (median 397)
Total size
1 TB+… See the full description on the dataset page: https://huggingface.co/datasets/pravsels/arx5-robot-dataset.robot_pickup_datasetThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "so101_follower",
"total_episodes": 50,
"total_frames": 10207,
"total_tasks": 1,
"total_videos": 0,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 30,
"splits": {
"train": "0:50"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path": null,
"features": {… See the full description on the dataset page: https://huggingface.co/datasets/DozenDucc/robot_pickup_dataset.robot-bottle-datasetThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "panda",
"total_episodes": 331,
"total_frames": 140760,
"total_tasks": 2,
"total_videos": 0,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 15,
"splits": {
"train": "0:331"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/jhalwaysthespecial/robot-bottle-dataset.chris_robot_dataset_with_tracesThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "panda",
"total_episodes": 4,
"total_frames": 1272,
"total_tasks": 1,
"total_videos": 0,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 15,
"splits": {
"train": "0:4"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/brandonyang/chris_robot_dataset_with_traces.robot_dataRobotData
RobotData — procedural-drawer demonstrations
Expert demonstrations for the RoboSynChallenge drawer_open_place task, collected
in simulation on procedurally generated drawer assets rather than the single
hand-authored drawer the benchmark ships with.
The task is bimanual: the right arm opens the drawer, the left arm picks a tomato
and places it inside, the right arm pushes the drawer shut.
Open the drawer, pick the tomato with the other arm then place it in the drawer.… See the full description on the dataset page: https://huggingface.co/datasets/wenjingbian/RobotData.robot-arena-benchmark-datacombined_chris_robot_datasetThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "panda",
"total_episodes": 0,
"total_frames": 0,
"total_tasks": 0,
"total_videos": 0,
"total_chunks": 0,
"chunks_size": 1000,
"fps": 15,
"splits": {},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/brandonyang/combined_chris_robot_dataset.real_robot_data_arrangement_mixedrobot-bottle-datasetThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "panda",
"total_episodes": 331,
"total_frames": 140760,
"total_tasks": 2,
"chunks_size": 1000,
"fps": 15,
"splits": {
"train": "0:331"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path": "videos/{video_key}/chunk-{chunk_index:03d}/file-{file_index:03d}.mp4"… See the full description on the dataset page: https://huggingface.co/datasets/michios/robot-bottle-dataset.gpt6_robotdatarobot_jersey_dataset
NUpbr RoboCup Jersey-Colour Robot Detection Dataset
1,707 physically-based-rendered images of RoboCup Humanoid Soccer scenes,
generated with NUpbr, each
containing 1-4 robots and one ball, with per-instance bounding boxes and,
for each robot, a jersey colour.
Every frame is restricted to at most two distinct jersey colours across all
robots present, regardless of robot count. Each robot is independently
assigned one of the two colours sampled fresh for that frame from a wide… See the full description on the dataset page: https://huggingface.co/datasets/Ysobel/robot_jersey_dataset.sim-robot-dataIB_Robot_dataset_banana_pick
数据集：1arm_2cam_banana_pick_v1_20260514
创建日期：2026-05-14
任务类型：banana_pick
任务描述：1臂2相机抓取香蕉放到盘子里
数据版本：v1
数据来源：实机采集（正常轨迹录制）
关联训练中间模型：暂无
关联可用模型：暂无
数据集详情
机器人类型：so_follower
相机数量：2（top, wrist）
总 episode 数：80
总帧数：45358
FPS：30
图像分辨率：480x640
视频编码：av1
动作维度：6（shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper）
Episode 分布
所有数据均为正常轨迹录制，香蕉初始位置按以下方式分布在盘子周围：
一、香蕉横放（episode 0~39）
Episode 范围
香蕉初始位置
0 ~ 9
盘子左侧
10 ~ 19… See the full description on the dataset page: https://huggingface.co/datasets/openEuler/IB_Robot_dataset_banana_pick.omx_f_Sugok_Robot_Arm_DataThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "omx_f",
"total_episodes": 47,
"total_frames": 11988,
"total_tasks": 1,
"total_videos": 94,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 15,
"splits": {
"train": "0:47"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/dream-79/omx_f_Sugok_Robot_Arm_Data.trossen-robot-data
Trossen Robot Cube Stacking Dataset
Dataset for training robotic manipulation policies on the Trossen WidowX Dual Robot cube stacking task using MuJoCo simulation.
Dataset Overview
This dataset contains demonstrations for a cube stacking task with delta actions, suitable for:
Behavioral Cloning (BC): Dual-arm 14D action space
Reinforcement Learning (RL): Single-arm 7D action space
RL Single-Arm Dataset (rl_delta_single_arm)
File:… See the full description on the dataset page: https://huggingface.co/datasets/poolvarine/trossen-robot-data.robot-trajectories-datasetchris_robot_dataset_with_traces_fixedThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "panda",
"total_episodes": 87,
"total_frames": 19814,
"total_tasks": 6,
"total_videos": 0,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 15,
"splits": {
"train": "0:87"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/brandonyang/chris_robot_dataset_with_traces_fixed.data
Dataset Card for TRANSIC Data
This dataset card is accompanied with the CoRL 2024 paper titled TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correction.
It includes generated simulation data and real-robot human correction data for sim-to-real transfer of robotic arm manipulation policies.
Dataset Details
Dataset Description
This dataset includes two parts, 1) simulation data used in student policy distillation and 2) real-robot data used in… See the full description on the dataset page: https://huggingface.co/datasets/transic-robot/data.