Datasetpublic

rovolabs/so-arm101-pick-place

Reproduction and extension of GPU-DAD's SO-101 Pick-Cube Dataset with MuJoCo This repository is our reproduction of the environment behind gpudad/so101_pick_cube, GPU-DAD's SO-101 pick-and-place data set. We rebuilt that scene in MuJoCo and used the replica to generate a 2,000-episode pick-and-place dataset of our own: rovolabs/so-arm101-pick-place. The original Environment This is a MuJoCo replica of the scene in gpudad/so101_pick_cube by gpudad. All fidelity… See the full description on the dataset page: https://huggingface.co/datasets/rovolabs/so-arm101-pick-place.

sourceHugging Facecc-by-4.0updated 23d agoView on Hugging Face

0likes162downloads

Dataset Card

Reproduction and extension of GPU-DAD's SO-101 Pick-Cube Dataset with MuJoCo

This repository is our reproduction of the environment behind gpudad/so101_pick_cube, GPU-DAD's SO-101 pick-and-place data set. We rebuilt that scene in MuJoCo and used the replica to generate a 2,000-episode pick-and-place dataset of our own: rovolabs/so-arm101-pick-place.

The original Environment

This is a MuJoCo replica of the scene in gpudad/so101_pick_cube by gpudad. All fidelity numbers below are measured against frames from that dataset.

[image]

Episode 44, frame 16 — from the original [gpudad/so101_pick_cube](https://huggingface.co/datasets/gpudad/so101_pick_cube) data set

The Replicated Environment

[image]

The same episode and frame, rendered here, in our replicated world

The comparison shows the synchronized front, overhead, and wrist camera views. The three replicated scenes differ in paint and lighting only; their geometry and physics are identical. During a rollout, one scene supplies the physics state and that state is mirrored into the camera-specific scenes for rendering.

The environment source, MJCF files, IK expert, and runnable examples are in Rovo-Labs/so-arm101-pick-place.

The Dataset

A companion dataset of 2,000 episodes of the pick-and-place cube task, recorded entirely in this replicated environment. It contains 1,000 successful episodes, where the cube ends up in the bin, and 1,000 failed episodes retained as labelled negative examples.

Episode range	Episodes	Controller	Success	Failure
0–99	100	GPU-DAD's recorded action data, replayed in our replicated world	100	0
100–999	900	Our inverse-kinematics expert, acting in our replicated world	500	400
1000–1999	1,000	A SmolVLA policy trained on GPU-DAD's own data, acting in our replicated world	400	600
Total	2,000		1,000	1,000

All three controller groups use the same language command:

text

Pick up the red cube and place it in the blue box.

Dataset details

Property	Value
Robot	SO-101 (6-DoF arm with gripper)
Episodes	2,000
Successful episodes	1,000
Failed episodes	1,000
FPS	30
Cameras	3 (front, overhead, wrist)
Resolution	512 × 512 RGB
Environment	MuJoCo reproduction of the GPU-DAD SO-101 pick-cube scene
Format	LeRobot v3.0

Controller labels

annotation.action_source and episode_success are constant within each episode.

Value	Label	Episodes	Success	Failure	Meaning
0	`ACTION_GPUDAD`	100	100	0	GPU-DAD train100 actions replayed in the replicated environment
1	`ACTION_IK`	900	500	400	Scripted inverse-kinematics expert attempts
2	`ACTION_SMOLVLA`	1,000	400	600	SmolVLA trained on GPU-DAD and rolled out in the replicated environment

episode_success is the evaluator-confirmed outcome and is constant within each episode. The balanced 1,000-success/1,000-failure composition is curated and must not be interpreted as the natural success rate of either controller.

Cameras

—Front camera: external view of the arm and workspace
—Overhead camera: top-down view of the cube, bin, and workspace
—Wrist camera: camera mounted near the gripper for close-up manipulation

The three camera streams are frame-aligned with the state and action rows.

State and action space

Both observation.state and action are six-element absolute joint-coordinate vectors in this exact order:

text

[elbow_flex, gripper, shoulder_lift, shoulder_pan, wrist_flex, wrist_roll]

Frame-level fields

Field	Type	Description
`episode_index`	`int64`	Dataset episode index
`frame_index`	`int64`	Frame index within the episode
`timestamp`	`float32`	Episode-relative time in seconds
`observation.state`	`float32[6]`	Current absolute robot joint coordinates
`action`	`float32[6]`	Absolute joint-position command
`annotation.action_source`	`int64[1]`	Controller-source enum: 0, 1, or 2
`episode_success`	`bool[1]`	Evaluator-confirmed success or failure outcome
`observation.images.front`	video, RGB 512 × 512	Front camera
`observation.images.overhead`	video, RGB 512 × 512	Overhead camera
`observation.images.wrist`	video, RGB 512 × 512	Wrist camera
`index`	`int64`	Global frame index
`task_index`	`int64`	Language-task lookup; always 0

Per-episode asset placement metadata

The starting cube and bin poses are stored once per episode in meta/episodes/chunk-000/file-000.parquet. They are episode metadata rather than frame-level policy observations.

Field	Type	Description
`placement_label`	`string`	Stable source placement identifier
`cube_starting_position_xyz_m`	`float64[3]`	Cube starting position `[x, y, z]` in metres
`cube_starting_orientation_quaternion_wxyz`	`float64[4]`	Cube starting orientation `[w, x, y, z]`
`bin_starting_position_xyz_m`	`float64[3]`	Bin starting position `[x, y, z]` in metres
`bin_starting_orientation_quaternion_wxyz`	`float64[4]`	Bin starting orientation `[w, x, y, z]`

Usage with LeRobot

python

from lerobot.datasets.lerobot_dataset import LeRobotDataset

dataset = LeRobotDataset("rovolabs/so-arm101-pick-place")
print(f"Episodes: {dataset.num_episodes}")
print(f"Frames: {dataset.num_frames}")

sample = dataset[0]
print(sample["annotation.action_source"], sample["episode_success"])

Use annotation.action_source and episode_success to construct controller-specific or outcome-specific subsets. The annotation fields are descriptive metadata and are not policy observations unless an experiment intentionally treats them that way.

License and Attribution

This dataset is licensed under CC BY 4.0. The 100 replayed GPU-DAD action trajectories originate from gpudad/so101_pick_cube, which is distributed under Apache 2.0.

Citation

If you use this dataset, please cite it:

bibtex

@dataset{so-arm101-pick-place_2026,
  author = {Rasheed, Armaan and Gallimore, Kilas and Johnson, Evonne and Subbiah, Vishal},
  license = {CC-BY-4.0},
  month = aug,
  title = {Reproduction and extension of {GPU-DAD}'s {SO-101} Pick-Cube Dataset with {MuJoCo}},
  url = {https://huggingface.co/datasets/rovolabs/so-arm101-pick-place},
  version = {1.0.0},
  year = {2026}
}

Environment source: Rovo-Labs/so-arm101-pick-place