fastwalker1118/b1k-task5-mousetraps

task-05 · VLA successes + mined nav demos (LeRobot v3.0)

Merged training set for setting_mousetraps: the short successful policy rollouts plus the navigation-recovery demonstrations mined from failed ones.

fps 30 · robot_type R1Pro · 1,780,548 frames · single task setting_mousetraps

Features

Depth streams and observation.cam_rel_poses are not included. π₀-style training consumes images + state + action only, and cam_rel_poses (3 cameras × pos+quat) is a forward-kinematics function of the 256-dim state. Both remain in by_instance/.

Everything is 224×224 because the mined demos were recorded at that resolution; the successes were downscaled from 720×720 (head) / 480×480 (wrist) to match.

Frames and units — read before training on velocities

All velocities in `observation.state` are exactly as the simulator reported them. No frame conversion is applied. This is deliberate: at eval the harness feeds the policy extract_state_from_proprio(obs["robot_r1::proprio"]), which slices base_qvel straight out of live proprioception. Training data must therefore carry the same quantity, or the policy learns to rely on a signal it never receives at deployment.

What that means concretely, for anyone reading these columns as physics:

• —base_qvel (253:256) and joint_qvel[0,1] (84,85) are the holonomic base virtual-joint velocities. Their frame is the articulation root, a fixed scene anchor — for this scene rotated 75.028 deg from world, with origin offset [0.2431, 2.0411]. They are NOT world frame and NOT robot frame.
• —robot_lin_vel (152:155) / robot_ang_vel (155:158) come from the root link and ARE world frame. So the two velocity blocks sit in frames 75 deg apart.
• —All of these are instantaneous values sampled at the tick boundary, not the average over the tick. With physics at 120 Hz and actions at 30 Hz, a stiff position-controlled joint has largely settled by the sample instant: integrating base_qvel recovers ~48% of the distance actually travelled, and the arm blocks are worse. Position columns are the smooth, self-consistent ones.

If you need true robot-frame velocity, derive it from base_xy/base_yaw odometry (backward difference) rather than rotating these columns — but do it on BOTH the training data and the eval wrapper, or you reintroduce the skew.

Note the observation and the action use different frames and different time semantics. That asymmetry is the benchmark's, not this dataset's.

The success episodes remain byte-identical to the challenge parquets they came from.

Note the model's own input is the 23-dim vector from extract_state_from_proprio (baseqvel 3 + trunkqpos 4 + armleft 7 + Lgrip 1 + armright 7 + Rgrip 1), not the 256-dim state stored here. Only dims 0-2 of that input are velocity; arm_*_qvel and trunk_qvel are never read by the policy.

Encoding

Every video is h264 / yuv420p (CRF 18), matching what info.json declares and what rollout_success uses. Both source halves needed correcting to get there: the mined demos were recorded as h264 yuv444p (High 4:4:4 — poor browser and NVDEC support), and the downscaled successes were first written as MPEG-4 Part 2. Frame counts were verified per file after re-encoding, so images stay aligned with actions.

Where the two halves come from

VLA successes — of the 498 evaluated rollouts, 66 reached q_score.final == 1.0. The 37 whose simulator_time was under 6 minutes are included. Note the outcome directory in by_instance/ is not a reliable success marker: early-terminating successes are filed under success/, but the 8 that ran to the 20344-step horizon are filed under failure/ despite q_score == 1.0. Success is decided only by logs/metrics/*.json.

Mined nav demos — recovery motions mined from failed rollouts by seeding the sim to the failure frame, retrieving a stance from the task-5 demo bank, and planning a tuck + forward-arc-RRT transit to it. Recorded with MOP_VLA_OFF=1, so each episode ends at the VLA handoff: these are motion demonstrations, and none has been shown to convert through a policy. 585 mined from 137 seeds over 15,612 attempts; median episode 15.2 s.

Duplication is physical

Episodes are repeated as separate episode_index values over identical frames, each with its own index range and its own video file — the same convention rollout_success uses for its natively-successful instances. This upweights nav demos roughly 1.7:1 against successes in a fixed-epoch run. It adds no new information: the 10.42 h of nav data is 2.6 h seen four times.

meta/episodes carries _source_kind, _source and _copy so the 622 unique episodes can be recovered without regenerating anything.

Layout note

One episode per video file (from_timestamp 0, to_timestamp length/30), rather than packing several episodes per file as rollout_success does. Readers locate frames by (chunk, file, from_timestamp, to_timestamp), which this satisfies.

Verified

Video frame count equals action row count for all 622 source episodes, re-checked after re-encoding; global index contiguous over 0..1,780,547; per-episode frame ranges contiguous; 2,414 video files present per camera; every video decodes as h264/yuv420p.