huzheyuan/abc130k_v3_train

ABC-130k — 256² full-frame re-conversion (train split)

Full-resolution re-conversion of the ABC-130k dual-YAM bimanual manipulation corpus (XDOF/ABC-130k raw recordings, Apache-2.0) in LeRobot v3.0 format.

This split: 129,032 episodes · 381,742,382 frames · 197 tasks · 30 fps. Validation split: huzheyuan/abc130k_v3_val (1,592 episodes / 189 tasks, task strings aligned to this split).

Why another port

The existing LeRobot port (lerobot/abc_130k_v3_train) letterboxes the 16:10 / 4:3 source streams into 224×224, so only 224×140 (stereo stations) or 224×168 (RealSense stations) of each frame is active — 30–40 % of every image is black bars. This conversion decodes the original camera streams (1920×1200 H.265 stereo stations, 640×480 H.264 RealSense stations) and squash-resizes the full frame to 256×256 (SVT-AV1, CRF 30): no letterboxing, ~30 % more active pixels per axis.

Lowdim data was converted from the same raw streams and validated against the LeRobot port tick-for-tick (episode alignment count-exact; full-trajectory joint max |Δ| = 1.2e-7 rad on both station types).

Differences vs the lerobot port

• —4 tasks absent (193 episodes): mix the coffee capsules…, dust the bookshelf…, put the gloves into the storage basket, tie the trash bag — these were removed from the raw corpus after the port was made, so no source video exists for them.
• —Frames are full-image 256×256 (squashed), not 224×224 letterboxed.
• —Extra features: joint_state / joint_action (14-dim raw joint layout) alongside the 20-dim Cartesian observation.state / action.

Features

Conventions: Cartesian pose is forward kinematics at the wrist flange (the corpus mixes three gripper hardware types with no per-episode label, so no TCP offset is applied); rotation is 6D (first two rotation-matrix columns); canonical +z = approach axis, y = jaw axis; per-arm base frames (REP-103, +x task-forward); grippers are fraction-open in [0, 1] (1 = open) in all four vectors.

Subtask annotations (subtasks episode-metadata column)

meta/episodes/*.parquet carries a subtasks column: a JSON-encoded list of {"text", "start", "end"} segments per episode, with frame indices [start, end) on this dataset's 30 fps frame grid. Present for 42,980 / 129,032 episodes (176 / 197 tasks); null for episodes without annotation. Segment boundaries were recovered from the raw XDOF/ABC-130k annotation.mcap streams on the exact conversion frame grid; labels are kept verbatim, including "mistake", "reset both arms" and "adjust ..." segments. Segments tile each annotated episode contiguously (65 episodes have an unlabeled prefix before the first segment).

import json
segs_json = ds.meta.episodes[idx]["subtasks"]  # None if unannotated
if segs_json is not None:
    segs = json.loads(segs_json)  # [{"text": ..., "start": 0, "end": 172}, ...]

Usage

from lerobot.datasets.lerobot_dataset import LeRobotDataset

ds = LeRobotDataset("huzheyuan/abc130k_v3_train")
item = ds[0]  # dict with images (CHW float32), state, action, ...

Loading note: timestamp tolerance

LeRobotDataset builds its video-timestamp queries in float32. Past ~1000 s into a packed v3 video file this misses the exact 30 fps grid by ~1e-4 s, which equals the default tolerance_s = 1e-4 and raises FrameTimestampError on perfectly valid data (the frame grid itself is exact; this affects every large packed v3 dataset, not just this one). Work around it by passing a slightly larger tolerance:

ds = LeRobotDataset("huzheyuan/abc130k_v3_train", tolerance_s=1e-3)

License

Apache-2.0, same as the source corpus. If you use this dataset, please also credit the original ABC-130k release (XDOF).