datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
PhysicalAI-Robotics-Locomanipulation-GRAIL
📢 News
[2026-07-15] Released task-general tracking policy checkpoints trained on the released data. Follow the tracking doc to use them to track our released motion data.
[2026-07-14] Updated data/pickup_table and data/pickup_ground. If you downloaded them before this date, please re-download.
Dataset Overview
Tabletop Pickup
Ground Pickup
Tabletop Manipulation
Ground Manipulation
Sitting
Curb
Slope… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-Locomanipulation-GRAIL.PhysicalAI-SimReady-Warehouse-01
NVIDIA Physical AI SimReady Warehouse OpenUSD Dataset
Dataset Version: 1.1.0
Date: May 18, 2025
Author: NVIDIA, Corporation
License: CC-BY-4.0 (Creative Commons Attribution 4.0 International)
Contents
This dataset includes the following:
This README file
A CSV catalog that enumerates all of the OpenUSD assets that are part of this dataset including a sub-folder of images that showcase each 3D asset (physical_ai_simready_warehouse_01.csv). The CSV file is organized in… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-SimReady-Warehouse-01.PhysicalAI-WorldModel-Synthetic-Physical-Interaction-Scenes
PhysicalAI-WorldModel-Synthetic-Physical-Interaction-Scenes Dataset Card
Dataset Description
PhysicalAI-WorldModel-Synthetic-Physical-Interaction-Scenes is a large-scale synthetic dataset of physically-simulated multi-object interaction scenes, generated using NVIDIA Isaac Sim and the PhysX physics engine. It is designed to train and evaluate AI models on physical reasoning, rigid body dynamics, optical flow, depth estimation, and scene understanding.
Each clip… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-WorldModel-Synthetic-Physical-Interaction-Scenes.Nemotron-Personas-Korea
Nemotron-Personas-Korea
우리나라 실제 분포에 기반한 합성 페르소나를 위한 복합 AI 시스템
A compound AI approach to personas grounded in real-world distributions
데이터셋 개요 (Overview)
Nemotron-Personas-Korea는 대한민국의 실제 인구통계학적·지리적·성격 특성 분포를 기반으로 합성된 오픈소스 페르소나 데이터셋(CC BY 4.0)으로, 우리나라 인구의 다양성과 특성을 폭넓게 반영하도록 설계되었습니다. 이는 최초의 대규모 우리말 페르소나 데이터셋이며, 이름, 성별, 나이, 혼인 상태, 교육 수준, 직업, 거주 지역 등의 속성을 실제 대한민국 국가데이터처 국가통계포털(KOSIS), 대법원, 국민건강보험공단, 농촌경제연구원, NAVER Cloud 통계 자료를 기반으로 합성하였습니다.… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Nemotron-Personas-Korea.PhysicalAI-Robotics-NuRec
Dataset Description
The Physical AI NuRec dataset seeks to empower robotic researchers to build the next generation of physical AI based end-to-end robotic models.
This dataset includes various 3DGUT in USD files that can be loaded in Isaac Sim. Some datasets also include a mesh and occupancy map. The Mesh components are used for collision detection while the 3DGUT components provide realistic rendering. The asset can also be used with Isaac Sim Extensions like MobilityGen for… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-NuRec.Nemotron-Personas-Japan
Nemotron-Personas-Japan
現実世界の分布に基づいたペルソナ生成のための複合AIアプローチ
データセット概要 (Dataset Overview)
Nemotron-Personas-Japan は、日本における人口の多様性と豊かさを捉えることを目的とし、実世界の人口統計、地理的分布、性格特性の分布に基づいて合成的に生成されたペルソナのオープンソースデータセットです。名前、性別、年齢、背景、婚姻状況、学歴、職業、居住地などの統計に基づいて生成した初のデータセットされた Nemotron-Personas の日本語版です。本バージョンでは、日本語における多様なモデリングユースケースに適した高品質のペルソナを提供します
Nemotron-Personas-Japan は、日本のモデル開発者が重要な地域固有の人口統計や文化的背景を取り入れたソブリンAIシステムを開発することを支援します。本データセットは、日本の地理的・人口統計的な実分布を反映することで、合成データの多様性を高め、バイアスを軽減し、model… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Nemotron-Personas-Japan.describe-anything-dataset
Describe Anything: Detailed Localized Image and Video Captioning
NVIDIA, UC Berkeley, UCSF
Long Lian, Yifan Ding, Yunhao Ge, Sifei Liu, Hanzi Mao, Boyi Li, Marco Pavone, Ming-Yu Liu, Trevor Darrell, Adam Yala, Yin Cui
[Paper] | [Code] | [Project Page] | [Video] | [HuggingFace Demo] | [Model/Benchmark/Datasets] | [Citation]
Dataset Card for Describe Anything Datasets
Datasets used in the training of describe anything models (DAM).
The datasets are in tar files. These… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/describe-anything-dataset.PhysicalAI-VANTAGE-Bench
VANTAGE-BENCH
Video ANalysis Tasks Across Generalized Environments
Paper: VANTAGE-Bench: Evaluating the Infrastructure AI Gap in Vision-Language Models
Dataset Description
VANTAGE-BENCH is the first public benchmark purpose-built for evaluating visual understanding on video captured by fixed infrastructure cameras. It spans three real-world domains — warehouse, smart city / Intelligent Transportation Systems (ITS), and smart spaces — across six spatio-temporal… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-VANTAGE-Bench.Nemotron-Personas-India
Nemotron-Personas-India
A compound AI approach to personas grounded in real-world distributions
वास्तविक दुनिया के वितरण पर आधारित व्यक्तित्वों के लिए एक मिश्रित AI दृष्टिकोण
Dataset Overview (डेटासेट अवलोकन)
Nemotron-Personas-India is an open-source (CC BY 4.0) dataset of synthetically-generated personas. This dataset is grounded in real-world demographic, geographic and personality trait distributions in India to capture the diversity and richness of… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Nemotron-Personas-India.omni-dreams-samples
AlpaDreams Samples
Curated single-view driving sequences for evaluating the
nvidia/alpadreams-dit world model.
Layout
data/
└── single_view/
├── <clip-id>/
| ├── <clip-id_...>.mp4 # ground truth video
│ ├── <clip-id_..._hdmap>.mp4 # HD-map rasterized conditioning video
│ ├── first_frame.png # RGB first frame, extracted from ground truth video
│ └── prompt.txt # text prompt
└──… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/omni-dreams-samples.ERFGS-Nvidiamiracl-vision
MIRACL-VISION
MIRACL-VISION is a multilingual visual retrieval dataset for 18 different languages. It is an extension of MIRACL, a popular text-only multilingual retrieval dataset. The dataset contains user questions, images of Wikipedia articles and annotations, which article can answer a user question. There are 7898 questions and 338734 images. More details can be found in the paper MIRACL-VISION: A Large, multilingual, visual document retrieval benchmark.
This dataset is ready… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/miracl-vision.Harmonizer-Dataset
HARMONIZER DATASET
Dataset Description
Training dataset for DiffusionHarmonizer: a generative AI model for image and video enhancement bridging neural reconstruction and photorealistic simulation .
Model checkpoints: https://huggingface.co/nvidia/Harmonizer/Training code: https://github.com/NVIDIA/harmonizer/
The dataset was curated to support the following functions of the model:
3D reconstruction artifact removal
Harmonization of inserted objects to blend… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Harmonizer-Dataset.Nemotron-Personas-Vietnam
Nemotron-Personas-Vietnam
Hệ thống AI kết hợp để tạo personas tổng hợp dựa trên phân bố thực tế của Việt Nam
A compound AI approach to personas grounded in real-world distributions
Tổng quan (Overview)
Nemotron-Personas-Vietnam là tập dữ liệu personas được cung cấp dưới dạng mã nguồn mở (CC BY 4.0) dựa trên phân bố nhân khẩu học, địa lý và đặc điểm tính cách của người Việt Nam. Tập dữ liệu phản ánh một cách toàn diện sự phong phú và đặc trưng… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Nemotron-Personas-Vietnam.Nemotron-Personas-France
Nemotron-Personas-France
Une approche d'IA composée pour des personas ancrés dans des distributions réelles
A compound AI approach to personas grounded in real-world distributions
Vue d'ensemble du jeu de données (Dataset Overview)
Nemotron-Personas-France est un jeu de données en libre accès (CC BY 4.0) composé de personas générés de manière synthétique. Ce jeu de données s'appuie sur les distributions démographiques, géographiques et de traits de… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Nemotron-Personas-France.ffs_stereo4d
FFS Stereo4D
[Project Page] [Paper] [Code]
Disparity maps for stereo matching, generated from the Stereo4D dataset using FoundationStereo.
Dataset Structure
data/train/
metadata.csv
0000000.zip (first 50,000 images)
0000001.zip (next 50,000 images)
...
0000025.zip
Each zip contains disparity PNG files named {vid_id}_frame_{frame_idx:06d}.png.
Disparity images: 3-channel uint8 784×784 PNG files encoding per-pixel disparity. Decode with: disp = (R *… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/ffs_stereo4d.PhysicalAI-Robotics-GR00T-Eval
EVAL-175
Dataset Description:
123 initial frame pictures from the robot's perspective before performing various tasks in the lab.
This dataset is ready for commercial/non-commercial use.
Dataset Owner(s):
NVIDIA Corporation (GEAR Lab)
Dataset Creation Date:
May 1, 2025
License/Terms of Use:
This dataset is governed by the Creative Commons Attribution 4.0 International License (CC-BY-4.0).
This dataset was created using a… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-GR00T-Eval.video-to-data-object-assets
NVIDIA Video-to-Data Object Assets
NVIDIA Video-to-Data Object Assets is a collection of textured 3D object meshes and
simulation-oriented USD packages produced for the
NVIDIA Video-to-Data project.
The repository is hosted as a Hugging Face dataset for versioned distribution, but its contents
are 3D assets rather than raw recordings, an annotated machine-learning dataset, or a benchmark.
Version and contents
This is the V2D v0.3 object-asset release.… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/video-to-data-object-assets.QCalEval
QCalEval
Dataset Description
The dataset contains scientific plots from quantum computing calibration experiments, paired with vision-language question-answer (QA) pairs. The dataset is used to evaluate a model's ability to interpret, classify, and reason about experimental results.
The dataset contains quantum computing calibration experiment data:
309 PNG plot images (scatter plots, line plots, heatmaps showing quantum device measurement results)
243 benchmark entries… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/QCalEval.Cosmos-AnomalyGen-PCB-Dataset
Dataset Overview
Dataset Description:
We are releasing 86 sample images of components on NV boards in a FoxConn factory. These images are used to finetune the AnomalyGen and Qwen-Image-Edit.
This dataset is for demonstration purposes and not for production usage.
Dataset Owner(s):
NVIDIA Corporation
Dataset Creation Date:
05/30/2026
Version:
1.0
License/Terms of Use:
GOVERNING TERMS: By downloading or… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Cosmos-AnomalyGen-PCB-Dataset.Spark-AnomalyGen-USD
Dataset Overview
Dataset Description:
The asset in question is the USD along with the components. Full PCBA scene (spark_lighting.usd) with an authored AOI ring-light rig (aoi_ring_light.usda) and camera — ready for synthetic data generation rendering. USD is derived from the underlying CAD design.
Dataset Owner(s):
NVIDIA Corporation
Dataset Creation Date:
05/30/2026
Version:
1.0
License/Terms of Use:… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Spark-AnomalyGen-USD.DLC-Bench
Describe Anything: Detailed Localized Image and Video Captioning
NVIDIA, UC Berkeley, UCSF
Long Lian, Yifan Ding, Yunhao Ge, Sifei Liu, Hanzi Mao, Boyi Li, Marco Pavone, Ming-Yu Liu, Trevor Darrell, Adam Yala, Yin Cui
[Paper] | [Code] | [Project Page] | [Video] | [HuggingFace Demo] | [Model/Benchmark/Datasets] | [Citation]
Dataset Card for DLC-Bench
Dataset for detailed localized captioning benchmark (DLC-Bench).
LICENSE
CC BY-NC-SA 4.0… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/DLC-Bench.Linear-Radiation-Transport
Dataset Description:
A surrogate-modeling dataset for the 2-D linear
Radiation Transport Equation (RTE), covering two canonical benchmarks
that vary along complementary axes:
Lattice (707 samples, 494 train / 106 val / 107 test) — fixed
7 × 7 block geometry; per-sample variation in the white-background
scattering coefficient σsW\sigma_s^WσsW and the blue-absorber cross-
section σaB\sigma_a^BσaB drawn from a discrete design grid (see
§ Data generation). QoI: final-time absorption… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Linear-Radiation-Transport.OCR-nvidia-Nemotron-VLM-Dataset-v2_wiki_fr-clean
Description
This dataset is a processed version of nvidia/Nemotron-VLM-Dataset-v2 to make it easier to use, particularly for a visual question answering task where answer is an OCR transcription.Specifically, the original dataset has been processed to provide the image directly as a PIL rather than a path in an image column.We've also translated question column to French containing 40 prompts based on via tutoiement, vouvoiement and imperative forms.
For further details, please… See the full description on the dataset page: https://huggingface.co/datasets/lbourdois/OCR-nvidia-Nemotron-VLM-Dataset-v2_wiki_fr-clean.nvidia-physical-ai-keyframes-sample
Dataset Card for 2025.11.20.21.48.24.136991
This is a FiftyOne dataset with 1000 samples.
Installation
If you haven't already, install FiftyOne:
pip install -U fiftyone
Usage
import fiftyone as fo
from fiftyone.utils.huggingface import load_from_hub
# Load the dataset
# Note: other available arguments include 'max_samples', etc
dataset = load_from_hub("dgural/nvidia-physical-ai-keyframes-sample")
# Launch the App
session = fo.launch_app(dataset)… See the full description on the dataset page: https://huggingface.co/datasets/dgural/nvidia-physical-ai-keyframes-sample.Shopify-product-catalogue-8kPhysicalAI-Robotics-mindmap-GR1-Drill-in-Box
Dataset Description:
This dataset is a multimodal collection of trajectories generated in Isaac Lab on the Drill in Box task defined in mindmap.
The task was created to evaluate robot manipulation policies on their spatial memory capabilities.
With this (partial) dataset you can generate the full dataset used for mindmap model training,
run a mindmap training or evaluate mindmap open/closed loop.
This dataset is for research and development only.
Dataset Owner(s):… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-mindmap-GR1-Drill-in-Box.PhysicalAI-Robotics-mindmap-Franka-Cube-Stacking
Dataset Description:
This dataset is a multimodal collection of trajectories generated in Isaac Lab on the Cube Stacking task defined in mindmap.
The task was created to evaluate robot manipulation policies on their spatial memory capabilities.
With this (partial) dataset you can generate the full dataset used for mindmap model training,
run a mindmap training or evaluate mindmap open/closed loop.
This dataset is for research and development only.
Dataset Owner(s):… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-mindmap-Franka-Cube-Stacking.nvidialab_recogida2This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"shape": [
6
],
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"… See the full description on the dataset page: https://huggingface.co/datasets/BravoRobots/nvidialab_recogida2.PhysicalAI-Robotics-mindmap-Franka-Mug-in-Drawer
Dataset Description:
This dataset is a multimodal collection of trajectories generated in Isaac Lab on the Mug in Drawer task defined in mindmap.
The task was created to evaluate robot manipulation policies on their spatial memory capabilities.
With this (partial) dataset you can generate the full dataset used for mindmap model training,
run a mindmap training or evaluate mindmap open/closed loop.
This dataset is for research and development only.
Dataset Owner(s):… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-mindmap-Franka-Mug-in-Drawer.