Datasetpublic

chomeed/mimicgen_square_d0_224x224_mtdit_flow_40k_failure_seg

MimicGen square_d0 failures — tail cut at the outcome Every episode of chomeed/mimicgen_square_d0_224x224_mtdit_flow_40k_failure with its tail cut at the outcome. The approach is kept in full — every clip starts at frame 0. Same task, same two cameras, same 20 fps, same feature schema. All 100 episodes kept, 21145 frames, median clip 176 of 400 source frames (44%). Where the tail is cut The outcome frame depends on how far the attempt got. All four cases are… See the full description on the dataset page: https://huggingface.co/datasets/chomeed/mimicgen_square_d0_224x224_mtdit_flow_40k_failure_seg.

sourceHugging Faceapache-2.0updated 20d agoView on Hugging Face

0likes215downloads

Dataset Card

MimicGen `square_d0` failures — tail cut at the outcome

Every episode of `chomeed/mimicgen_square_d0_224x224_mtdit_flow_40k_failure` with its tail cut at the outcome. The approach is kept in full — every clip starts at frame 0. Same task, same two cameras, same 20 fps, same feature schema.

All 100 episodes kept, 21145 frames, median clip 176 of 400 source frames (44%).

Where the tail is cut

The outcome frame depends on how far the attempt got. All four cases are labelled per episode in meta/segment_info.json as end_kind:

`end_kind`	n	median clip len	median endpoint residual (mm)
`lifted_then_failed`	80	173	51.6
`never_reached_nut`	11	400	347.1
`failed_grasp`	9	79	343.2

`end_kind`	cut at
`rewarded`	first frame with `reward > 0` (none here — all episodes are failures)
`lifted_then_failed`	`argmin` over frames at/after the grasp of `\	nut − ref\	` — closest approach to the seated pose
`failed_grasp`	`argmin` of `\	eef − nut\	` at/after the grasp attempt — the moment the grasp failed
`never_reached_nut`	last frame; nothing to cut at, episode kept whole

Failed grasps are included and cut properly. A grasp attempt is fingers closing within 80 mm of the nut with no sustain requirement — a failed grasp by definition does not stay near the nut, so the usual "sustained over 40 frames" test rejects exactly the cases worth keeping. Of the 20 episodes where the nut never left the table, 9 are real grasp attempts (cut at median 79 frames instead of trailing 400) and 11 never brought the gripper to the nut at all.

ref = [228.8, 101.3, 887.2] mm, the seated nut pose from `..._40k_success` (std [3.3, 4.2, 9.9] mm — the peg is fixed in square_d0, so threading's relative |(needle − tripod) − ref| collapses to |nut − ref| in world coordinates).

Per-episode outcome labels

Independently of where the clip is cut, meta/segment_info.json records what happened to the nut, derived from state: {'dropped': 30, 'neverlifted': 20, 'releasedelevated': 45, 'heldatend': 5}.

outcome	meaning
`released_elevated`	ends elevated but far from an open gripper — left on the peg, unseated
`dropped`	lifted >30 mm, then back on the table
`never_lifted`	nut never leaves the table (splits into `failed_grasp` / `never_reached_nut` above)
`held_at_end`	still in the gripper at timeout

Detection uses lift off the table, not a grasp threshold: a nut cannot leave the table unless the gripper lifts it. Two calibration details that are easy to get wrong:

—The nut spawns at z = 0.890 and settles to ~0.822, so a global table height reads a phantom 68 mm "lift" on frame 0 of every episode. The baseline here is per-episode, after settling.
—|nut − eef| bottoms out at 41 mm — it is nut-centre to gripper, while a square nut is grasped by its rim. Tighter thresholds find nothing.

observation.state (23 dims) ships without per-key features; layout recovered empirically: [0:3] eef_pos, [3:7] eef_quat, [7:9] gripper_qpos, [9:12] nut_pos, [12:16] nut_quat, [16:19] nut_to_eef_pos, [19:23] nut_to_eef_quat.

Caveat

Endpoint residual is only meaningful for lifted_then_failed (median 51.6 mm). For failed_grasp and never_reached_nut the nut never approaches the peg, so their ~345 mm residual is a description of that fact, not a miss distance. Those two kinds are a different failure mode from "tried to insert and missed" and may deserve separate treatment as reward-model negatives.