Datasetpublic

khang123452/GR1-Manipulation-IsaacSim-Rendered

GR1 Manipulation — Isaac Sim Rendered Dataset Dataset Description Multi-modal rendered dataset of NVIDIA GR1 humanoid robot performing tabletop manipulation tasks in Isaac Sim. Each episode is rendered as 3 synchronized MP4 videos: RGB — ray-traced color video Depth — normalized depth map (grayscale, 0–10m range) Segmentation — semantic segmentation with per-object class coloring This dataset is designed as input for NVIDIA Cosmos Transfer 2.5 to generate… See the full description on the dataset page: https://huggingface.co/datasets/khang123452/GR1-Manipulation-IsaacSim-Rendered.

sourceHugging Facecc-by-4.0updated 6mo agoView on Hugging Face

0likes78downloads

Dataset Card

GR1 Manipulation — Isaac Sim Rendered Dataset

Dataset Description

Multi-modal rendered dataset of NVIDIA GR1 humanoid robot performing tabletop manipulation tasks in Isaac Sim. Each episode is rendered as 3 synchronized MP4 videos:

—RGB — ray-traced color video
—Depth — normalized depth map (grayscale, 0–10m range)
—Segmentation — semantic segmentation with per-object class coloring

This dataset is designed as input for NVIDIA Cosmos Transfer 2.5 to generate photorealistic sim-to-real augmented training data for robot learning.

Dataset Structure

{task_name}/
  episode_NNNN_rgb.mp4      # 1280x720, 20fps, RayTracedLighting
  episode_NNNN_depth.mp4     # 1280x720, 20fps, normalized depth
  episode_NNNN_seg.mp4       # 1280x720, 20fps, semantic segmentation

Tasks

#	Task Name	Episodes	Frames/Episode
1	`PnPBottleToCabinetClose`	458	~120–400
2	`PnPCanToDrawerClose`	2	~120–400
3	`PnPCupToDrawerClose`	2	~120–400
4	`PnPMilkToMicrowaveClose`	2	~120–400
5	`PnPPotatoToMicrowaveClose`	2	~120–400
6	`PnPWineToCabinetClose`	2	~120–400
7	`PosttrainPnPNovelFromCuttingboardToBasketSplitA`	2	~120–400
8	`PosttrainPnPNovelFromCuttingboardToCardboardboxSplitA`	2	~120–400
9	`PosttrainPnPNovelFromCuttingboardToPanSplitA`	2	~120–400
10	`PosttrainPnPNovelFromCuttingboardToPotSplitA`	2	~120–400
11	`PosttrainPnPNovelFromCuttingboardToTieredbasketSplitA`	2	~120–400
12	`PosttrainPnPNovelFromPlacematToBasketSplitA`	2	~120–400
13	`PosttrainPnPNovelFromPlacematToBowlSplitA`	1	~120–400
14	`PosttrainPnPNovelFromPlacematToPlateSplitA`	2	~120–400
15	`PosttrainPnPNovelFromPlacematToTieredshelfSplitA`	2	~120–400
16	`PosttrainPnPNovelFromPlateToBowlSplitA`	2	~120–400
17	`PosttrainPnPNovelFromPlateToCardboardboxSplitA`	1	~120–400
18	`PosttrainPnPNovelFromPlateToPanSplitA`	2	~120–400
19	`PosttrainPnPNovelFromPlateToPlateSplitA`	1	~120–400
20	`PosttrainPnPNovelFromTrayToCardboardboxSplitA`	2	~120–400
21	`PosttrainPnPNovelFromTrayToPlateSplitA`	2	~120–400
22	`PosttrainPnPNovelFromTrayToPotSplitA`	2	~120–400
23	`PosttrainPnPNovelFromTrayToTieredbasketSplitA`	1	~120–400
24	`PosttrainPnPNovelFromTrayToTieredshelfSplitA`	2	~120–400

Recording Settings

Setting	Value
Camera	`egoview` (1st-person, attached to `robot0_head_pitch`)
FOV	90° vertical
Resolution	1280×720
FPS	20
Renderer	Isaac Sim RayTracedLighting, SPP=32
Lighting	Dome light, intensity=1500
USD Export	robosuite exporter, geomgroup=[0,1,0,0,0,0] (visual meshes only)

Source Data

—Robot: NVIDIA GR1 humanoid (from gr00trobosuite)
—Tasks: RoboCasa GR1 tabletop pick-and-place tasks
—Simulation: MuJoCo → USD → Isaac Sim Replicator pipeline
—Original trajectories: 1000 episodes per task in HDF5 format

Intended Use

Input for Cosmos Transfer 2.5 world generation model to produce photorealistic augmented manipulation videos for robot policy training (sim-to-real transfer).

License

CC-BY-4.0