rubatotree/classify-blocks-512-local

classify-blocks 512

A LeRobot v3.0 dataset for one long-horizon pick-and-place task: a block is picked from a compartment of a transparent organiser, carried to the matching compartment that shows the same block class, and released. The snapshot holds 512 episodes / 201,004 frames at 15 fps and 640x480 RGB from a wrist-mounted camera. Robot type is so_follower; the six action and state channels are shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll (degrees) and gripper (percent). action is an absolute position command, not a velocity.

Composition

20 episodes are measured teleoperation from a single recording session. The remaining 492 episodes are synthesised: a recorded grasp is replayed in a physically reconstructed scene, the block layout, the organiser pose and the whole appearance stack are randomised, and the result is rendered in Blender Cycles at 64 samples per pixel. Synthetic episodes carry randomized compartment occupancy (including empty compartments and stacked blocks), randomized class inventory, and randomised organiser yaw so the policy sees the placement error of a real table. Appearance randomisation varies the class surface (clear plastic through metal-filled plastic), the illumination, the environment and the camera mount, with 12 explicitly extreme illumination families: dim calm light, cold low light, hard raking light, split-colour key, hotspot, shadow sweep, blown key, four-hue mix, many small sources, a travelling key, hard cast shadow and a hue sweep. Light motion is a native location curve, so a "moving light" episode really moves.

Layout and features

The dataset follows the LeRobot v3.0 layout: per-episode tables under data/chunk-000/, one aggregated videos/observation.images.front/chunk-000/file-XXX.mp4 per chunk, and episode metadata under meta/episodes/. Besides the standard keys there is one extra column, training.action_valid (int64). It is a supervision mask: 1 where the recorded command is a valid target, 0 where the command should not be imitated. It is not a policy input.

Provenance and caveats

The measured episodes all come from one session with one operator and one organiser, so this snapshot makes no real-world generalisation claim; source_counts and the full per-episode audit chain live in the local snapshot manifest, and the local provenance/ tree, snapshot_manifest.json and validation.json are intentionally not published here. The synthetic half is a geometric and photometric approximation of the real setup: joint values and contacts are simulated, and only the appearance is checked against the recording, so the synthetic frames should be treated as an augmentation of the measured episodes rather than as independent ground truth.

Usage

from lerobot.datasets.lerobot_dataset import LeRobotDataset

dataset = LeRobotDataset("rubatotree/classify-blocks-512-local")

The single task string is Match each loose plastic block to the examples in the organizer and put it in the matching compartment.