andlyu/Play_with_blocks_combined

Play with blocks — reviewed demonstrations

LeRobot v3.0 dataset of bimanual YAM demonstrations manipulating colored blocks, chips and a container. Gemini proposed time-aligned action labels, followed by human review and selection of retained sections.

Contents

• —23,329 frames at 30 Hz (12 minutes 57.63 seconds).
• —25 continuous episodes from 13 source demonstrations.
• —Three synchronized RGB views: observation.images.top, .left, .right.
• —14-dimensional absolute joint-pose actions and observations.
• —Joint order: six left joints, six right joints, left gripper, right gripper.
• —task_index and subtask_index resolve the reviewed action instruction.
• —source_episode_index and source_frame_index preserve original provenance.

Selection and labels

This release uses the first 14 human-reviewed demonstrations from the labeled selection. One was entirely excluded, leaving 13 contributing demonstrations. All sections explicitly marked “Leave out” are removed, as is one unlabeled final frame. Failed actions may remain when the reviewer retained them; labels can explicitly describe failure, incomplete motion or uncertainty.

Each uninterrupted retained span becomes one episode. Instructions vary per frame through taskindex/subtaskindex. Action chunks can cross instruction boundaries, but cannot bridge omitted frames or original source episode boundaries. See meta/continuousepisodes.json for the current output episode mapping; reviewedsourcemanifest.json retains the earlier action-span provenance. The full review snapshots and source-frame mappings are in `meta/reviewedsource_manifest.json`.

Loading

Use a LeRobot v3-compatible loader, for example LeRobot 0.5.1:

from lerobot.datasets.lerobot_dataset import LeRobotDataset

dataset = LeRobotDataset(
    "andlyu/Play_with_blocks_combined",
    revision="main",
    video_backend="pyav",
)

All 25 output episodes are assigned to the default train split. For evaluation, split by original source_episode_index, not by individual action episode, to avoid leakage between adjacent segments from the same demonstration.

Validation

The export was loaded with LeRobot; task/subtask labels, camera decoding and source indices were checked at continuous episode boundaries. Every retained action and state row was compared exactly with the original source data. See validation.json. Labels remain subject to normal human annotation uncertainty.

This is the reviewed dataset in its original joint ordering, not the separately prepared MolmoAct2 training copy. BimanualYAM checkpoint consumers must reorder joints and grippers to match their model contract.