datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Hy-Embodied-0.5-VLA-Data
Hy-Embodied-0.5-VLA
From Vision-Language-Action Models to a Real-World Robot Learning Stack
Tencent Robotics X × Tencent Hy Team
📖 Abstract
We introduce Hy-Embodied-0.5-VLA (Hy-VLA) — an end-to-end Vision-Language-Action system that spans the full robot learning stack: data collection, model design, pre-training, supervised fine-tuning, RL post-training, and real-world deployment. Built on the Hy-Embodied-0.5 MoT backbone, Hy-VLA integrates a flow-matching… See the full description on the dataset page: https://huggingface.co/datasets/tencent/Hy-Embodied-0.5-VLA-Data.AirScape-Dataset
[ACM MM'25] AirScape: An Aerial Generative World Model with Motion Controllability
This repository contains the dataset introduced in the paper, consisting of two parts: 11k+ motion intention prompts and corresponding video clips.
Arxiv: https://arxiv.org/pdf/2507.08885
Project: https://embodiedcity.github.io/AirScape/
Code: https://github.com/EmbodiedCity/AirScape.code
Dataset Description
This dataset is proposed for training and testing of aerial world models.… See the full description on the dataset page: https://huggingface.co/datasets/EmbodiedCity/AirScape-Dataset.UrbanVideo-Bench
[ACL'25 Oral] UrbanVideo-Bench: Benchmarking Vision-Language Models on Embodied Intelligence with Video Data in Urban Spaces
This repository contains the dataset introduced in the paper, consisting of two parts: 5k+ multiple-choice question-answering (MCQ) data and 1k+ video clips.
Arxiv: https://arxiv.org/pdf/2503.06157
Project: https://embodiedcity.github.io/UrbanVideo-Bench/
Code: https://github.com/EmbodiedCity/UrbanVideo-Bench.code
Dataset Description
The… See the full description on the dataset page: https://huggingface.co/datasets/EmbodiedCity/UrbanVideo-Bench.vsi-benchEmbodied-R1-Dataset
Embodied-R1-3B-v1
Embodied-R1: Reinforced Embodied Reasoning for General Robotic Manipulation (ICLR 2026)
[🌐 Project Website] [📄 Paper] [🏆 ICLR2026 Version] [🎯 Dataset] [📦 Code]
Model Details
Model Description
Embodied-R1 is a 3B vision-language model (VLM) for general robotic manipulation.
It introduces a Pointing mechanism and uses Reinforced Fine-tuning (RFT) to bridge perception and action, with strong zero-shot generalization in embodied… See the full description on the dataset page: https://huggingface.co/datasets/IffYuan/Embodied-R1-Dataset.EmbodiedAgentInterface
Dataset Card for Embodied Agent Interface
This is the dataset used in our project Embodied Agent Interface.
EmbodiedNav-Bench
[KDD'26 Oral] How Far Are Large Multimodal Models from Human-Level Spatial Action? A Benchmark for Goal-Oriented Embodied Navigation in Urban Airspace
EmbodiedNav-Bench is a goal-oriented embodied navigation benchmark for evaluating spatial action in urban 3D airspace. The benchmark contains 5,037 high-quality navigation trajectories with natural-language navigation goals, initial drone poses, target positions, and ground-truth 3D trajectories.
This Hugging Face repository… See the full description on the dataset page: https://huggingface.co/datasets/EmbodiedCity/EmbodiedNav-Bench.pc_gpu_franka_ik.jointtarget_20hz_20260906This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 20,
"features": {
"observation.images.front": {
"dtype": "video",
"shape": [
480,
640,
3
],
"names": [
"height",
"width",
"channels"
],
"info": {
"video.height":… See the full description on the dataset page: https://huggingface.co/datasets/EmbodiedSWE/pc_gpu_franka_ik.jointtarget_20hz_20260906.EmbodiedVerse-BenchARCHIEVED-behavior-1k-embodiedAI-rollouts-hq-use
BEHAVIOR-1K Task-0 — Demos + X-VLA Rollouts
A merged dataset for BEHAVIOR-1K Task 0 (turning_on_radio) that combines:
200 human-teleoperated demonstrations from the upstream HuggingFace
behavior-1k/2025-challenge-demos (HF rows, episode indices 10..3000 stride-10).
259 X-VLA policy rollouts collected locally on the same scenes via OmniGibson, using the
Hoshipu/xvla-behavior1k-v20 task0-60k checkpoint
(collected rows, episode indices 5000..5258). All 259 are successful completions… See the full description on the dataset page: https://huggingface.co/datasets/Hoshipu/ARCHIEVED-behavior-1k-embodiedAI-rollouts-hq-use.clear_organic_agent_ab_B1000.jointtarget_20hz_20260918This dataset was created using LeRobot.
About this dataset (agent-in-the-loop A/B, arm B, first 1000 verified successes)
Simulated Franka Panda demonstrations of a produce-clearing task (pick every organic item - lemons, a lime, an
orange, a small pumpkin - off a cluttered packing table and drop it into the crate, leaving the distractors),
generated in Isaac Sim by the CoSiGen data-generation pipeline. Every episode is a certified success of the task
grader (all present organics… See the full description on the dataset page: https://huggingface.co/datasets/EmbodiedSWE/clear_organic_agent_ab_B1000.jointtarget_20hz_20260918.Embodied-CoTnut_thread_franka_ik.jointtarget_20hz_20260905This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 20,
"features": {
"observation.images.front": {
"dtype": "video",
"shape": [
480,
640,
3
],
"names": [
"height",
"width",
"channels"
],
"info": {
"video.height":… See the full description on the dataset page: https://huggingface.co/datasets/EmbodiedSWE/nut_thread_franka_ik.jointtarget_20hz_20260905.pc_ram_franka_osc.jointpos_15hz_20260907This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "assembly.pc_ram.franka.osc",
"total_episodes": 600,
"total_frames": 1053222,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 65,
"fps": 15,
"splits": {
"train": "0:600"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet"… See the full description on the dataset page: https://huggingface.co/datasets/EmbodiedSWE/pc_ram_franka_osc.jointpos_15hz_20260907.embodied-ai-conference-papers-2023-2025
Embodied AI Conference Papers 2023–2025
A dataset of 3,140 papers from 12 robotics conferences (CoRL 2023–2025, ICRA 2023–2025, IROS 2023–2025, RSS 2023–2025), processed with KNOWHERE.
Series
This collection of KNOWHERE-processed paper datasets is continuously expanding. New datasets and editions will be released — stay tuned!
Top ML Conference Papers
2023
https://huggingface.co/datasets/JensCS/top-ml-conference-papers-2023… See the full description on the dataset page: https://huggingface.co/datasets/JensCS/embodied-ai-conference-papers-2023-2025.clear_organic_agent_ab_B400.jointtarget_20hz_20260918This dataset was created using LeRobot.
About this dataset (agent-in-the-loop A/B, arm B, first 400 verified successes)
Simulated Franka Panda demonstrations of a produce-clearing task (pick every organic item - lemons, a lime, an
orange, a small pumpkin - off a cluttered packing table and drop it into the crate, leaving the distractors),
generated in Isaac Sim by the CoSiGen data-generation pipeline. Every episode is a certified success of the task
grader (all present organics… See the full description on the dataset page: https://huggingface.co/datasets/EmbodiedSWE/clear_organic_agent_ab_B400.jointtarget_20hz_20260918.pc_gpu_ram_franka_osc.jointpos_15hz_20260907This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "assembly.pc_gpu_ram.franka.osc",
"total_episodes": 600,
"total_frames": 1400130,
"total_tasks": 3,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 78,
"fps": 15,
"splits": {
"train": "0:600"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet"… See the full description on the dataset page: https://huggingface.co/datasets/EmbodiedSWE/pc_gpu_ram_franka_osc.jointpos_15hz_20260907.clear_organic_agent_ab_A1000.jointtarget_20hz_20260918This dataset was created using LeRobot.
About this dataset (agent-in-the-loop A/B, arm A, first 1000 verified successes)
Simulated Franka Panda demonstrations of a produce-clearing task (pick every organic item - lemons, a lime, an
orange, a small pumpkin - off a cluttered packing table and drop it into the crate, leaving the distractors),
generated in Isaac Sim by the CoSiGen data-generation pipeline. Every episode is a certified success of the task
grader (all present organics… See the full description on the dataset page: https://huggingface.co/datasets/EmbodiedSWE/clear_organic_agent_ab_A1000.jointtarget_20hz_20260918.pen_holder_franka_osc.jointpos_15hz_20260907This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "packing.pen_holder.franka.osc",
"total_episodes": 600,
"total_frames": 1057500,
"total_tasks": 2,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 44,
"fps": 15,
"splits": {
"train": "0:600"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet"… See the full description on the dataset page: https://huggingface.co/datasets/EmbodiedSWE/pen_holder_franka_osc.jointpos_15hz_20260907.datasetThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "so_follower",
"total_episodes": 160,
"total_frames": 81786,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:160"},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/Embodied-AI-6/dataset.pc_gpu_franka_osc.jointpos_15hz_20260907This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "assembly.pc_gpu.franka.osc",
"total_episodes": 600,
"total_frames": 1128960,
"total_tasks": 5,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 81,
"fps": 15,
"splits": {
"train": "0:600"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet"… See the full description on the dataset page: https://huggingface.co/datasets/EmbodiedSWE/pc_gpu_franka_osc.jointpos_15hz_20260907.clear_organic_objects_franka_ik.jointtarget_20hz_20260910This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 20,
"features": {
"observation.images.front": {
"dtype": "video",
"shape": [
480,
640,
3
],
"names": [
"height",
"width",
"channels"
],
"info": {
"video.height":… See the full description on the dataset page: https://huggingface.co/datasets/EmbodiedSWE/clear_organic_objects_franka_ik.jointtarget_20hz_20260910.tool_packing_franka_osc.jointpos_15hz_20260907This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "packing.tool_packing.franka.osc",
"total_episodes": 597,
"total_frames": 899343,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 24,
"fps": 15,
"splits": {
"train": "0:597"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet"… See the full description on the dataset page: https://huggingface.co/datasets/EmbodiedSWE/tool_packing_franka_osc.jointpos_15hz_20260907.coffee_franka_joint.jointtarget_60hz_20260907This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "puzzle.coffee.franka.joint",
"total_episodes": 600,
"total_frames": 1584630,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 62,
"fps": 60,
"splits": {
"train": "0:600"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet"… See the full description on the dataset page: https://huggingface.co/datasets/EmbodiedSWE/coffee_franka_joint.jointtarget_60hz_20260907.clear_organic_agent_ab_A400.jointtarget_20hz_20260918This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 20,
"features": {
"observation.images.front": {
"dtype": "video",
"shape": [
480,
640,
3
],
"names": [
"height",
"width",
"channels"
],
"info": {
"video.height":… See the full description on the dataset page: https://huggingface.co/datasets/EmbodiedSWE/clear_organic_agent_ab_A400.jointtarget_20hz_20260918.robotics-embodied-ai-physical-control-2026
🤖 Robotics, Embodied AI & Physical World Control Dataset (2026 Edition)
A structured research dataset featuring 2,123 domain-verified research papers and official code repositories focused on Vision-Language-Action Models (VLA), Humanoid Robotics, Quadruped Locomotion, Diffusion Policies, Sim-to-Real Transfer, and Physics Simulation Environments (Isaac Sim, MuJoCo, Genesis).
Built with Universal Scientific Engine V16.1 Gold, providing 43 schema attributes with verified… See the full description on the dataset page: https://huggingface.co/datasets/beatsprom/robotics-embodied-ai-physical-control-2026.blue_so101_color_blocks_10ep_v1_20260526_175604This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/Embodied-AI-6/blue_so101_color_blocks_10ep_v1_20260526_175604.open-eqabulb_franka_osc_jointpos_60hzThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 60,
"features": {
"observation.images.front": {
"dtype": "video",
"shape": [
480,
640,
3
],
"names": [
"height",
"width",
"channels"
],
"info": {
"video.height":… See the full description on the dataset page: https://huggingface.co/datasets/EmbodiedSWE/bulb_franka_osc_jointpos_60hz.so101_embodied_tasks_v1This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "so_follower",
"total_episodes": 100,
"total_frames": 32825,
"total_tasks": 2,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:100"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/fbarbosa1/so101_embodied_tasks_v1.