IDEAS-Lab-Northwestern/datagen-clutter-v1-joint-5cam

datagen-clutter-v1-joint-5cam

Auto-generated SFT dataset for the clutter (pick-out-of-clutter → place-in-goal) family — a cuRobo-planned, physics- & LTL-safety-checked demonstration set, already converted to LeRobot v2.1.

• —Creator: yypeng666 (IDEAS-Lab-Northwestern)
• —Source bench: IDEAS-Lab-Northwestern/ManiGuard-Bench — collected on all 55 `clutter_pickup` base tasks.
• —Per task: 40 success + LTL-safe trajectories → 2,200 episodes total.

Contents

Task

Pick the target object out of a cluttered tabletop and move it into the green goal sphere. Per-episode language prompt (meta/tasks.jsonl) is sourced from the base task's diagnostics, e.g. "Pick up the teacup on the desk, then move it into the green goal sphere …". Each demo must reach the goal AND stay LTL-safe (nothing toppled); failed/unsafe attempts are dropped.

Schema

State / actions (absolute joint; convert arm joints to per-step delta at train time):

Cameras (all 5 kept; pick a subset at train time):

image_opposite, image_left, image_right, image_left_shoulder (4 third-person) + wrist_image.

Notes

• —Camera subset is a train-time choice — all 5 streams ship here; a policy typically consumes one third-person view + wrist (2-cam) or +one more (3-cam).
• —Action variant — train on actions (b, next-achieved); actions_commanded (a) is extra.
• —The MimicGen sim-state replay dump is not included (kept in the raw archive; not needed for SFT).

License

Released under CC BY-NC 4.0 (see `LICENSE`). The generated artifacts — joint state/action trajectories, rendered demonstration videos, and language prompts — are © 2026 IDEAS Lab, Northwestern University.

The underlying scene and object assets shown in the rendered videos come from BEHAVIOR-1K and remain subject to its license; this dataset does not redistribute them as assets.

Paper & Citation

Part of ManiGuard: paper (arXiv:2608.17386) · code · docs

@misc{peng2026maniguard,
  title         = {{MANIGUARD}: A Benchmark and Data Suite for Specification-Grounded
                   Safety Evaluation and Improvement of Robotic Manipulation},
  author        = {Peng, Yiyan and Wang, Philip and Zhan, Simon Sinong and Lyu, Yiqi
                   and Ni, Zhenyang and Yan, Jixin and Wong, Fiorelli and Jiao, Ruochen
                   and Yin, Hang and Cao, Xinyu and Shao, Huajie and Li, Manling
                   and Zhang, Ruohan and Zhu, Qi},
  year          = {2026},
  eprint        = {2608.17386},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url           = {https://arxiv.org/abs/2608.17386},
}