Datasetpublic

cagataydev/strands-kitchen-stereo4d

🍳 Kitchen Stereo4D Dataset Stereo depth + spatial audio pick-and-place dataset generated by strands-robots on NVIDIA Jetson AGX Thor Overview This dataset contains synchronized stereo RGB, metric depth maps, 3D point clouds, and binaural spatial audio from robot manipulation tasks in a kitchen environment. Metric Value Episodes 10 Total Frames 501 Resolution 640×480 Stereo Baseline 50mm FPS 30 Robot Franka Emika Panda (simplified) GPU… See the full description on the dataset page: https://huggingface.co/datasets/cagataydev/strands-kitchen-stereo4d.

sourceHugging Faceapache-2.0updated 7mo agoView on Hugging Face

0likes2.6kdownloads

Dataset Card

🍳 Kitchen Stereo4D Dataset

Stereo depth + spatial audio pick-and-place dataset generated by [strands-robots](https://github.com/cagataycali/strands-gtc-nvidia) on NVIDIA Jetson AGX Thor

Overview

This dataset contains synchronized stereo RGB, metric depth maps, 3D point clouds, and binaural spatial audio from robot manipulation tasks in a kitchen environment.

Metric	Value
Episodes	10
Total Frames	501
Resolution	640×480
Stereo Baseline	50mm
FPS	30
Robot	Franka Emika Panda (simplified)
GPU	NVIDIA Thor (Blackwell, 132GB)
Simulator	MuJoCo 3.5.0

Generation Pipeline

Kitchen Scene (MuJoCo) → Stereo Cameras → Depth Estimation → Point Cloud
         ↓                                        ↓                ↓
  Contact Events → Spatial Audio Synthesis    Disparity Maps    Dataset
         ↓                                        ↓                ↓
  Robot Trajectory → State Recording        Visualization   HuggingFace

Structure

├── episode_000/
│   ├── stereo_left/        # Left camera RGB frames (PNG)
│   ├── stereo_right/       # Right camera RGB frames (PNG)  
│   ├── depth/              # Metric depth maps (NPY, float32, meters)
│   ├── disparity/          # Disparity maps (NPY, float32, pixels)
│   ├── point_clouds/       # 3D point clouds (NPY, float32, XYZ)
│   ├── audio/              # Binaural spatial audio (WAV, 44.1kHz)
│   ├── contacts.json       # Physics contact events
│   ├── robot_state.json    # Joint positions, velocities
│   └── metadata.json       # Episode metadata
├── ...
├── overview.png            # Scene overview render
├── side_view.png           # Side view render
├── metadata.json           # Dataset metadata
└── README.md               # This file

Episodes

Episode	Task	Frames	Contacts	Mode
0	Pick up the mug and place it on the plate	97	14918	gpu_mujoco
1	Pick up the bowl and move it to the cutting board	97	14810	gpu_mujoco
2	Grasp the spatula and place it next to the bowl	97	14954	gpu_mujoco
3	Exploration trajectory 1	30	6253	gpumujocovariation
4	Exploration trajectory 2	30	6350	gpumujocovariation
5	Exploration trajectory 3	30	6406	gpumujocovariation
6	Exploration trajectory 4	30	6830	gpumujocovariation
7	Exploration trajectory 5	30	6786	gpumujocovariation
8	Exploration trajectory 6	30	6325	gpumujocovariation
9	Exploration trajectory 7	30	6325	gpumujocovariation

Spatial Audio

Audio is synthesized from physics contact events with binaural spatialization:

—Ceramic objects (mug, plate, bowl): High-frequency impacts (2200Hz base)
—Wood objects (cutting board, spatula): Mid-frequency (800Hz base)
—Metal contacts (gripper): Sharp transients (4000Hz base)
—Spatial panning: Interaural Level Difference (ILD) based on contact position

Usage

python

import numpy as np
from PIL import Image

# Load a frame
left = np.array(Image.open("episode_000/stereo_left/frame_000000.png"))
right = np.array(Image.open("episode_000/stereo_right/frame_000000.png"))
depth = np.load("episode_000/depth/frame_000000.npy")
disparity = np.load("episode_000/disparity/frame_000000.npy")
point_cloud = np.load("episode_000/point_clouds/frame_000000.npy")

# Load audio
import soundfile as sf
audio, sr = sf.read("episode_000/audio/spatial_audio.wav")

# Load metadata
import json
with open("episode_000/metadata.json") as f:
    meta = json.load(f)

Citation

bibtex

@misc{strands_kitchen_stereo4d_2026,
    title={Kitchen Stereo4D: Pick-and-Place with Spatial Audio},
    author={strands-robots},
    year={2026},
    publisher={HuggingFace},
    url={https://huggingface.co/datasets/cagataydev/strands-kitchen-stereo4d},
    note={Generated on NVIDIA Jetson AGX Thor (Blackwell, 132GB)},
}

Hardware

—Device: NVIDIA Jetson AGX Thor
—GPU: Blackwell (sm_110), 132GB unified memory
—CUDA: 13.0
—PyTorch: 2.12.0+cu130
—MuJoCo: 3.5.0

—strands-robots — Agentic robotics framework
—nvidia/ffs_stereo4d — Reference stereo4D format
—nvidia/PhysicalAI-Robotics-Manipulation-Objects-Kitchen-MJCF