GT-111/robomme_v3_eef

RoboMME v3 EEF (LeRobot v3)

This is a LeRobot v3 format conversion of RoboMME, the first large-scale robotic benchmark dedicated to memory-augmented manipulation, reorganized into a canonical 8D end-effector (EE) pose representation. 20 Hz, single-arm Franka Panda, with cognitively motivated task suites covering pick-and-place, routing, pattern memory, and video-conditioned tasks.

Original Dataset

RoboMME: A Robotic Benchmark for Memory-Augmented Manipulation (ICML 2026 Spotlight)

RoboMME team. RoboMME: A Robotic Benchmark for Memory-Augmented Manipulation. arXiv:2603.04639, ICML 2026 (Spotlight).

• —Paper: https://arxiv.org/abs/2603.04639
• —Project Website: https://robomme.github.io/
• —GitHub: https://github.com/RoboMME/robommepolicylearning
• —Original Dataset (H5): https://huggingface.co/datasets/Yinpei/robommedatah5

RoboMME spans four cognitively motivated task suites with carefully designed tasks that push robots to remember, reason, and act — including tasks where the target is revealed only in an initial video demonstration (video* suites).

License

This dataset is released under Apache 2.0.

Conversion Details

What we changed

1. 1.Canonical EE Pose Representation: Per-frame proprioception is unified into a single 8D EE pose vector — position xyz (metres) + quaternion in scalar-first WXYZ order + gripper openness in metres (0.04 = fully open).

1. 1.Next-Observation-Absolute Actions: Actions are stored as 8D absolute EE pose commands. Row t stores the pose that the next observation row t+1 attains (next-observation-absolute alignment).

1. 1.LeRobot v3 Format: Converted to the LeRobot v3 dataset layout with sharded lossless H.264 video and Parquet-based frame data at 20 Hz.

What we preserved

• —Both camera views: observation.images.head (front view) and observation.images.wrist_right (wrist view), 256x256 RGB
• —Episode structure and task labels
• —Frame-level timestamps
• —observation.is_video_demo marks rows originating from video demos

Observation and action convention

• —observation.state: 8D end-effector pose — [x, y, z, qw, qx, qy, qz, gripper_openness_m].
• —action: 8D absolute end-effector pose command with the same layout; row t is the pose attained by observation row t+1.
• —The boolean column transition_action_valid is true exactly on rows whose action is a valid next-observation pose; terminal placeholder rows repeat the current observation state and are masked false.
• —Gripper channel is openness in metres: 0.04 = fully open.

Layout

Each task is a self-contained LeRobot v3 dataset in its own subdirectory with its own meta/, data/, and videos/:

robomme_v3_eef/
├── <task_name>/
│   ├── data/chunk-*/file-*.parquet             # Frame data (8D state + 8D action + validity)
│   ├── videos/
│   │   ├── observation.images.head/chunk-*/file-*.mp4
│   │   └── observation.images.wrist_right/chunk-*/file-*.mp4
│   └── meta/
│       ├── info.json                           # Dataset metadata
│       ├── tasks.parquet                       # Task vocabulary
│       ├── episodes/chunk-*/file-*.parquet     # Episode index
│       ├── stats.json                          # Per-feature dataset statistics
│       └── norm_stats.json                     # Per-task post-pipeline stats (schema v3)
└── ... (one directory per task)

Statistics

Usage

from lerobot.datasets import LeRobotDataset

# Each task is an independent LeRobot dataset under a subdirectory
dataset = LeRobotDataset("GT-111/robomme_v3_eef", root=".../<task_name>")

Citation

If you use this dataset, please cite the original RoboMME paper:

@article{robomme2026,
  title={RoboMME: A Robotic Benchmark for Memory-Augmented Manipulation},
  author={RoboMME team},
  journal={arXiv preprint arXiv:2603.04639},
  year={2026}
}

Version History

• —v3.0 (current): LeRobot v3 conversion with canonical 8D EE pose layout and next-observation-absolute actions

v2 mask semantics

Every child exposes observation.is_video_demo and transition_action_valid in both Parquet and meta/info.json. Demo rows are observation/memory context only; demo, terminal, and broken rows contribute to no action, future, progress, or normalization loss. The action contract is eef_absolute_next_observation_wxyz_v2.

Canonical action and demo contract v2

Every child uses eef_absolute_next_observation_wxyz_v2. On a valid transition, action[t] is the normalized, sign-continuous absolute WXYZ EEF state observed at t+1; the former physical controller command is not retained as the training action. observation.is_video_demo=true rows are memory/context only. Demo-boundary, terminal, and broken transitions have transition_action_valid=false and contribute to no action, future, progress, or normalization loss. The action repair leaves video frame payloads and Parquet timestamps unchanged; MP4 container timing is normalized to the declared 20 Hz cadence.