datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
PhysicalAI-Robotics-Manipulation-Kitchen
PhysicalAI Robotics Manipulation in the Kitchen
Dataset Description:
PhysicalAI-Robotics-Manipulation-Kitchen is a dataset of automatic generated motions of robots performing operations such as opening and closing cabinets, drawers, dishwashers and fridges. The dataset was generated in IsaacSim leveraging reasoning algorithms and optimization-based motion planning to find solutions to the tasks automatically [1, 3]. The dataset includes a bimanual manipulator built with… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-Manipulation-Kitchen.PhysicalAI-Robotics-GR00T-Teleop-G1
Unitree G1 Fruits Pick and Place 1K Dataset
Dataset Description:
The PhysicalAI-Robotics-GR00T-Teleop-G1 dataset consists of1000 teleoperation trajectories of real robot data using Unitree G1, with upper body control. The robot chooses the correct fruit to pick and place on the plate according to the language prompt. A total of 4 fruits are used: Apple, Pear, Starfruit, Grape. The robot is equipped with the default realsense camera, and a pair of Unitree G1 Tri-fingers… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-GR00T-Teleop-G1.physical-ai-bench-understanding
Physical AI Bench - Understanding
PAI-Bench (Physical AI Bench) is a comprehensive benchmark designed to evaluate physical AI generation and understanding capabilities across various real-world scenarios. This particular dataset, PAI-Bench-U, focuses specifically on Video Understanding tasks, comprising 2,808 real-world cases with task-aligned metrics.
Paper: PAI-Bench: A Comprehensive Benchmark For Physical AI
Code: GitHub Repository
Citation
If you use Physical AI… See the full description on the dataset page: https://huggingface.co/datasets/shi-labs/physical-ai-bench-understanding.PhysicalAI-Robotics-Manipulation-ObjectsPhysicalAI-Robotics-Manipulation-Objects is a dataset of automatic generated motions of robots performing operations such as picking and placing objects in a kitchen environment. The dataset was generated in IsaacSim leveraging reasoning algorithms and optimization-based motion planning to find solutions to the tasks automatically [1, 3]. The dataset includes a bimanual manipulator built with Kinova Gen3 arms. The environments are kitchen scenes where the furniture and appliances were… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-Manipulation-Objects.PhysicalAI-GR00T-Tuned-Tasks
Dataset Description:
This dataset is multimodal collections of trajectories generated in Isaac Lab. It supports humanoid (GR1) tabletop manipulation tasks for industrial settings. Each dataset entry provides the full context (state, vision, language, action) needed to train and evaluate generalist robot policies for tasks like pouring nuts or sorting pipes by color.
Dataset Name
# Trajectories
Exhaust-Pipe-Sorting-task
1000
Nut-Pouring-task
1000
This dataset is… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-GR00T-Tuned-Tasks.physical-ai
DecisionFacts Physical AI Dataset — SO-101 Robotic Arm Teleoperation
Data Summary
This dataset is a curated collection of real-world teleoperation data captured on the SO-101 robotic arm (so_follower), built to support training and evaluation of modern robot-learning models — from imitation-learning policies to large-scale Vision-Language-Action (VLA) and world models.
Each episode is a human-teleoperated demonstration of a manipulation task, recorded… See the full description on the dataset page: https://huggingface.co/datasets/DecisionFacts/physical-ai.physical-ai-bench-understanding-evalphysicalaiThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/slvrfivo/physicalai.Physical_AI_SO101_Cup_Nesting_Task
DecisionFacts Physical AI Dataset — SO-101 Robotic Arm Teleoperation
Data Summary
This dataset is a curated collection of real-world teleoperation data captured on the SO-101 robotic arm (so_follower), built to support training and evaluation of modern robot-learning models — from imitation-learning policies to large-scale Vision-Language-Action (VLA) and world models.
Each episode is a human-teleoperated demonstration of a manipulation task, recorded… See the full description on the dataset page: https://huggingface.co/datasets/salehin21/Physical_AI_SO101_Cup_Nesting_Task.so101_1200ep_dataset_20260803_104129This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/physicalairi/so101_1200ep_dataset_20260803_104129.physical_ai_tuneupPhysicalAI-Robotics-Manipulation-ObjectsPhysicalAI-Robotics-Manipulation-Objects is a dataset of automatic generated motions of robots performing operations such as picking and placing objects in a kitchen environment. The dataset was generated in IsaacSim leveraging reasoning algorithms and optimization-based motion planning to find solutions to the tasks automatically [1, 3]. The dataset includes a bimanual manipulator built with Kinova Gen3 arms. The environments are kitchen scenes where the furniture and appliances were… See the full description on the dataset page: https://huggingface.co/datasets/muhammadshihab/PhysicalAI-Robotics-Manipulation-Objects.physical-ai-bucharest-1This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "so_follower",
"total_episodes": 1,
"total_frames": 594,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:1"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/vladfatu/physical-ai-bucharest-1.PhysicalAI-SimReady-Homes
PhysicalAI SimReady Homes: Multi-Room Interiors
Version 1.0 · 1,000 simulation-ready multi-room home interiors that load into Isaac
Sim and are ready to train on — no rigging, no retopology, no physics authoring.
Every scene is a furnished multi-room home in OpenUSD with rigid bodies, mass and inertia,
collision approximations, PhysX friction/restitution/density, articulated doors and drawers,
per-object semantic labels, PBR materials, HDRI lighting and placed cameras — plus a… See the full description on the dataset page: https://huggingface.co/datasets/imagineio/PhysicalAI-SimReady-Homes.ur10e-gear-pick-place-demos
UR10e Gear Pick-and-Place Demonstrations
Private archive of the exact LeRobot v3.0 train and validation directories used by the revised Rho delta-action IL run. The task is to pick up the large gear and place it in the bin using a simulated UR10e with a Robotiq 2F-140 gripper.
📊 Dataset identity
Split
Episodes
Frames
Original data bytes
train/
90
154,593
277,712,776
validation/
10
17,170
30,883,658
Total
100
171,763
308,596,434
The corpus was… See the full description on the dataset page: https://huggingface.co/datasets/physical-ai-toolchain/ur10e-gear-pick-place-demos.ur3_stack_cube_camera_sim_v2This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"observation.state": {
"dtype": "float32",
"shape": [
12
],
"names": [
"shoulder_pan_joint.pos",
"shoulder_lift_joint.pos",
"elbow_joint.pos",
"wrist_1_joint.pos",
"wrist_2_joint.pos"… See the full description on the dataset page: https://huggingface.co/datasets/physicalairi/ur3_stack_cube_camera_sim_v2.physical-ai-evals-libero-spatial-pilot
LIBERO-Spatial paired pilot
Historical rollout records for OpenVLA and VLA-JEPA on 100 paired LIBERO-Spatial episode
specifications. The dataset contains 200 policy/episode records and 23,283 transition rows.
This is an exploratory trace, not a LIBERO reference evaluation or a confirmatory model
comparison. The artifact label 2026-07-02 is not a recorded execution timestamp.
Files
steps.parquet: normalized rollout-v1 transition rows.
episodes.parquet: one row per… See the full description on the dataset page: https://huggingface.co/datasets/Eventual-Inc/physical-ai-evals-libero-spatial-pilot.physical_AiThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "so_follower",
"total_episodes": 2,
"total_frames": 1004,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:2"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/young331/physical_Ai.so101_pick_orange_sim_v2This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "so101_follower",
"total_episodes": 45,
"total_frames": 46278,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:45"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/physicalairi/so101_pick_orange_sim_v2.PhysicalAI-SimReady-Kitchens-v1
PhysicalAI SimReady Kitchens: 800 Scenes
800 simulation-ready OpenUSD kitchen environments for robotics, embodied AI, and physical AI evaluation.
This release is a public sample of Imagine.io's programmable world-generation infrastructure. It is designed to help research and commercial teams evaluate controlled, metadata-rich indoor environments for perception, scene understanding, simulation, and physical AI workflows.
License notice. Public access is CC BY-NC 4.0 for… See the full description on the dataset page: https://huggingface.co/datasets/imagineio/PhysicalAI-SimReady-Kitchens-v1.edition_2130_nvidia-PhysicalAI-Robotics-GR00T-X-Embodiment-Sim-readymade
edition_2130_nvidia-PhysicalAI-Robotics-GR00T-X-Embodiment-Sim-readymade
A Readymade by TheFactoryX
Original Dataset
nvidia/PhysicalAI-Robotics-GR00T-X-Embodiment-Sim
Process
This dataset is a "readymade" - inspired by Marcel Duchamp's concept of taking everyday objects and recontextualizing them as art.
What we did:
Selected the original dataset from Hugging Face
Shuffled each column independently
Destroyed all row-wise relationships
Preserved structure… See the full description on the dataset page: https://huggingface.co/datasets/TheFactoryX/edition_2130_nvidia-PhysicalAI-Robotics-GR00T-X-Embodiment-Sim-readymade.physical-ai-hack-test-setedition_1235_nvidia-PhysicalAI-Robotics-GR00T-X-Embodiment-Sim-readymade
edition_1235_nvidia-PhysicalAI-Robotics-GR00T-X-Embodiment-Sim-readymade
A Readymade by TheFactoryX
Original Dataset
nvidia/PhysicalAI-Robotics-GR00T-X-Embodiment-Sim
Process
This dataset is a "readymade" - inspired by Marcel Duchamp's concept of taking everyday objects and recontextualizing them as art.
What we did:
Selected the original dataset from Hugging Face
Shuffled each column independently
Destroyed all row-wise relationships
Preserved structure… See the full description on the dataset page: https://huggingface.co/datasets/TheFactoryX/edition_1235_nvidia-PhysicalAI-Robotics-GR00T-X-Embodiment-Sim-readymade.