kisarakira/duobench_modify

DuoBench Modify

duobench_modify is an unofficial derivative of `RobotControlStack/duobench`. It combines the 11 DuoBench simulation subsets into one LeRobot v3 dataset, preserves the original joint-space fields, and adds 20-dimensional end-effector (EEF) state and action fields for TwinVLA-style training.

Scope: this repository contains simulation data only. The four upstream real-robot subsets are not included.

Dataset summary

Camera streams:

• —observation.images.head
• —observation.images.left_wrist
• —observation.images.right_wrist

Included simulation task groups:

• —ball_maze
• —bin_sort
• —block_balance
• —carry_pot
• —hinge_chest
• —join_blocks
• —pour_marbles
• —spring_door
• —transfer_cube
• —transfer_gate
• —transfer_reorient

The 11 task groups contain 12 distinct language instructions because carry_pot has two instructions.

Features

The EEF layout is left arm followed by right arm. Each arm uses:

[x, y, z, R[:, 0], R[:, 1], gripper]

The complete 20-dimensional order is:

left_xyz(3), left_rotation_6d(6), left_gripper(1),
right_xyz(3), right_rotation_6d(6), right_gripper(1)

R[:, 0] and R[:, 1] are the first two columns of the EEF rotation matrix. Positions and orientations are expressed in the shared DuoMount base frame.

EEF reconstruction

The added EEF labels were reconstructed from the original 16-dimensional joint vectors with forward kinematics using:

• —the FR3 kinematic model from RCS 0.7.2;
• —the inverse Robotiq 2F-85 TCP offset;
• —FR3_DUOMOUNT_LEFT_ROBOT and FR3_DUOMOUNT_RIGHT_ROBOT base transforms.

For both state and action, the seven joints of each arm are passed through forward kinematics, transformed into the shared DuoMount frame, converted to position plus 6D rotation, and concatenated with the original gripper value. The EEF fields are therefore derived labels rather than independently measured poses.

Video preservation

The 33 source MP4 files were copied one-to-one during aggregation. They were not decoded, concatenated, or re-encoded. Every destination MP4 was verified to be byte-identical to its corresponding source file.

Loading with LeRobot

from lerobot.datasets.lerobot_dataset import LeRobotDataset

dataset = LeRobotDataset("kisarakira/duobench_modify")

print(len(dataset))                  # 285988
print(dataset.meta.total_episodes)  # 550

sample = dataset[0]
print(sample["observation.state"].shape)      # (16,)
print(sample["action"].shape)                 # (16,)
print(sample["observation.state_eef"].shape)  # (20,)
print(sample["action_eef"].shape)             # (20,)
print(sample["observation.images.head"].shape)  # (3, 224, 224)

Validation

The generated dataset was reloaded through LeRobotDataset and checked for:

• —550 episodes, 285,988 frames, and 12 language instructions;
• —contiguous global frame and episode indices;
• —exact preservation of every original joint-state and joint-action row;
• —exact agreement of every EEF row with the precomputed FK output;
• —correct task-string remapping;
• —valid global and per-episode statistics, including EEF quantiles;
• —correct source-to-target video mapping at every task boundary;
• —byte-for-byte equality of all 33 source and destination MP4 files.

Limitations

• —Only the 11 simulation subsets are included. This is not the complete DuoBench release and contains no real-robot episodes.
• —The dataset inherits the filtering, 30 FPS timing, binary gripper labels, and other preprocessing choices of the upstream LeRobot release.
• —EEF values depend on the selected FR3 model, TCP offset, DuoMount transforms, and coordinate conventions.
• —action_eef is the FK projection of the upstream joint action, not a separately measured future EEF pose.
• —This repository is not an official release by the DuoBench authors and does not introduce benchmark results or policy-performance claims.

Source, license, and citation

This derivative follows the upstream dataset's Apache-2.0 license. Please also review the source dataset and cite the original DuoBench work:

• —Dataset: RobotControlStack/duobench
• —Raw dataset: RobotControlStack/duobench_raw
• —Code: RobotControlStack/duobench
• —Project page: duobench.github.io

@misc{duobench,
  title={{DuoBench}: A Reproducible Benchmark for Bimanual Manipulation in Simulation and the Real World},
  author={Tobias J{\"u}lg and Seongjin Bien and Simon Hilber and Yannik Blei and Pierre Krack and Maximilian Li and Sven Parusel and Rudolf Lioutikov and Florian Walter and Wolfram Burgard},
  year={2026},
  url={https://arxiv.org/abs/2606.11901}
}

中文说明

本数据集将 DuoBench 的 11 个仿真子集合并为一个 LeRobot v3 数据集，不包含 real 数据。它保留原始 16 维双臂关节状态与动作，并新增通过 FR3 正运动学重构的 20 维双臂 EEF 状态与动作。本仓库是非官方衍生版本，使用时请同时遵守上游许可证与引用要求。