Datasetpublic

Hoshipu/real_robot_data

Real Robot Manipulation Dataset Tabletop manipulation tasks recorded on a UR5e with a Robotiq 2F-140 gripper. Most tasks use two Intel RealSense cameras (fixed base + wrist-mounted); the newest hide_block_with_cup task adds a third (opposite-side counter view). All recordings at 20 Hz. Tasks Folder Eps Cameras Instruction put_blocks_in_plate 100 2 Pick two blocks from the table and place them on the plate. remove_blocks_from_plate 100 2 Pick two… See the full description on the dataset page: https://huggingface.co/datasets/Hoshipu/real_robot_data.

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face

0likes5.3kdownloads

Dataset Card

Real Robot Manipulation Dataset

Tabletop manipulation tasks recorded on a UR5e with a Robotiq 2F-140 gripper. Most tasks use two Intel RealSense cameras (fixed base + wrist-mounted); the newest hide_block_with_cup task adds a third (opposite-side counter view). All recordings at 20 Hz.

Tasks

Folder	Eps	Cameras	Instruction
`put_blocks_in_plate`	100	2	Pick two blocks from the table and place them on the plate.
`remove_blocks_from_plate`	100	2	Pick two blocks off the plate and place them on the table.
`hide_blocks_with_bowl`	100	2	Pair the two blocks together and cover them with the bowl.
`remove_blocks_under_bowl`	100	2	Uncover the hidden blocks and spread the three objects out.
`hide_blocks_with_cup`	100	2	Cover each block with an upside-down cup.
`put_blocks_on_cup`	100	2	Lift each cup off a block and stack the block on top of the cup.
`put_blocks_down`	100	2	Take the blocks off the cups and place them on the table.
`hide_block_with_cup`	120	3	Cover the block with the upside-down cup. (eps 1-66 light, 67-120 dark)

Each task folder includes a task_instruction.json with the full task spec (objects, pick/place strategies, gripper modes, workspace bounds).

`hide_block_with_cup` specifics: single-task variant (1 cup + 1 block). Recording captures only the cup-pick + cup-place motion; the inter-episode object randomization is not recorded. A 3rd camera (camera3, oppo-countertop) was added for this task. Per-episode meta.json includes a lighting field ("light" or "dark") — the room lighting changed at episode 67.

Layout

<task>/
    1.hdf5  ...  100.hdf5     # per-episode trajectories (images + state)
    meta/
        1/meta.json  ...  100/meta.json   # per-episode ground truth
    task_instruction.json     # task spec

HDF5 schema (per episode)

/timestamps                       (T,)      float64
/observations/
    images/
        camera1                   (T,H,W,3) uint8   # base, 640x480 RGB
        camera2                   (T,H,W,3) uint8   # wrist, 640x480 RGB
        camera3                   (T,H,W,3) uint8   # (hide_block_with_cup only) oppo-countertop
    qpos                          (T,7)     float32 # 6 joints + gripper (norm)
    state                         (T,14)    float32 # tcp_pos(3)+tcp_quat(4)+joints(6)+gripper(1)
    tcp_pos, tcp_quat, joints, gripper_pos

The HDF5 root attribute num_cameras reports the actual count for that file (2 for all 7 original tasks, 3 for hide_block_with_cup).

meta.json schema (per episode)

For all tasks the meta contains the per-episode ground-truth object positions:

json

{
    "task": "<task name>",
    "episode": <int>,
    "start": {<objects>: <(x,y)>, ...},
    "end":   {<objects>: <(x,y)>, ...},
    "init_pose": [x, y, z]
}

Task-specific keys are additionally recorded (e.g. perm and cup_order for the cup tasks, sign and active/passive for the bowl hide task, block_order for the bowl/cup remove tasks).

Robot setup

—Robot: UR5e via RTDE
—Gripper: Robotiq 2F-140 (auto-calibrated)
—TCP orientation: pointing straight down (axis-angle [π, 0, 0])
—Cameras: 2× Intel RealSense, 640×480 @ 20 Hz
—Workspace: trapezoid with y ∈ [0.50, 0.80] and x half-width 0.10 → 0.18 (linearly with y)

Per-task episode counts

Task	Episodes	Frames (~)	Wall-clock (~)
putblocksin_plate	100	45,576	38 min
removeblocksfrom_plate	100	45,929	38 min
hideblockswith_bowl	100	43,881	37 min
removeblocksunder_bowl	100	66,090	55 min
hideblockswith_cup	100	(~varies)	~40 min
putblockson_cup	100	(~varies)	~50 min
putblocksdown	100	(~varies)	~30 min

License

Apache 2.0.