CoRL2026-CSI/SO101-cap_stack_RGBblock_on_bluedish_10fps

SO101 CAP Stack RGB Blocks on Blue Dish

This dataset contains 100 LeRobot v3.0 demonstration episodes for an SO101 follower robot. The task is: Stack red, green, and blue blocks on the blue dish from bottom to top. The dataset was collected at 10 Hz and includes paired top-view and wrist-view RGB videos, robot state/action trajectories, and CAP skill annotations.

<a class="flex" href="https://huggingface.co/spaces/lerobot/visualizedataset?path=CoRL2026-CSI/SO101-capstackRGBblockonbluedish10fps"> <img class="block dark:hidden" src="https://huggingface.co/datasets/huggingface/badges/resolve/main/visualize-this-dataset-xl.svg"/> <img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface/badges/resolve/main/visualize-this-dataset-xl-dark.svg"/> </a>

Dataset Details

Task And Annotations

Task: Stack red, green, and blue blocks on the blue dish from bottom to top.

Task indices: 0

Observed subtask annotations:

• —pick red block and place on blue dish
• —pick green block and place on red block
• —pick blue block and place on green block

Representative skill types:

• —move_initial
• —moveandopen
• —move
• —gripper_close
• —gripper_open
• —moveandclose
• —move_free

Representative skill-language annotations:

• —move to initial state
• —Approach red block and open gripper
• —Pick up the red block
• —grasp red block
• —Lift the red block
• —Move red block above blue dish
• —Place red block on blue dish
• —release object on blue dish
• —Retreat from blue dish and close gripper
• —Approach green block and open gripper
• —Pick up the green block
• —grasp green block

Observation And Action Space

Raw camera keys are observation.images.top and observation.images.left_wrist. The local SmolVLA training scripts map them to observation.images.camera2 and observation.images.camera1, respectively.

Files

meta/info.json
meta/tasks.parquet
meta/episodes/chunk-*/file-*.parquet
data/chunk-*/file-*.parquet
videos/{observation.images.top,observation.images.left_wrist}/chunk-*/file-*.mp4

The dataset uses the LeRobot v3.0 format. Episode metadata and frame-level trajectories are stored in parquet files; image observations are stored as H.264 MP4 videos referenced by the frame records.

Annotation Columns

Loading

from lerobot.datasets.lerobot_dataset import LeRobotDataset

dataset = LeRobotDataset("CoRL2026-CSI/SO101-cap_stack_RGBblock_on_bluedish_10fps")
sample = dataset[0]

Intended Uses

This dataset is intended for robot imitation learning, action-chunking policy training, skill-conditioned behavior analysis, and reproducible LeRobot/SmolVLA experiments on the specified tabletop task.

Limitations

The dataset is task-specific and collected in a fixed workspace. It does not include an official validation or test split, nor does it include benchmark success-rate results. Downstream users should verify camera calibration, action normalization, and task-language assumptions before transferring policies to a different robot, workspace, or object set.