Datasetpublic

Voxel51/DreamTac

DreamTac → FiftyOne (Native Multimodal MCAP) DreamTac from Peking University, converted to native multimodal MCAP episodes. A Franka Emika Panda works through contact-rich tabletop tasks while four cameras record on one 20 fps clock: a third-person view, a wrist view, and two Xense Photon vision-based tactile sensors mounted on the gripper fingertips. The fingertips are the point of the release. Each is a gel pad printed with a marker grid, and the grid deforms where the object… See the full description on the dataset page: https://huggingface.co/datasets/Voxel51/DreamTac.

sourceHugging Facecc-by-4.0updated 20d agoView on Hugging Face

1likes1.7kdownloads

Dataset Card

DreamTac → FiftyOne (Native Multimodal MCAP)

[image]

DreamTac from Peking University, converted to native multimodal MCAP episodes.

A Franka Emika Panda works through contact-rich tabletop tasks while four cameras record on one 20 fps clock: a third-person view, a wrist view, and two Xense Photon vision-based tactile sensors mounted on the gripper fingertips. The fingertips are the point of the release. Each is a gel pad printed with a marker grid, and the grid deforms where the object presses, so the moment of contact is visible rather than inferred from a force reading.

Installation

bash

pip install fiftyone

Usage

python

import fiftyone as fo
import fiftyone.utils.huggingface as fouh

dataset = fouh.load_from_hub(
    "Voxel51/DreamTac",
    name="DreamTac",
    persistent=True,
)
fo.launch_app(dataset)

The trajectories that moved the furthest:

python

view = dataset.sort_by("end_effector_path_m", reverse=True)
fo.launch_app(dataset, view=view)

What you get

703 trajectories across 12 tasks, 506,078 frames, 7.03 hours. Every frame carries all four views, so the set holds 2,024,312 camera frames in total.

Each episode contains:

—/camera, the third-person view at 224x224, as foxglove.CompressedVideo
—/wrist-camera, the same from the wrist
—/tactile-left and /tactile-right, the two Xense Photon fingertips
—/end-effector-pose, position and orientation in the Franka base frame, as foxglove.PoseInFrame
—/end-effector.plot, the same as x, y, z and the rotation vector
—/gripper.plot, the gripper opening
—/instruction, the task text, written at each change so a task with sub-steps shows them on the timeline

Task	Episodes	Frames	Minutes	Mean path	Instruction
`pick_bread`	99	105,603	88.0	1.83 m	Pick up the baguette.
`clean_whiteboard`	104	86,843	72.4	1.11 m	Clean the whiteboard with an eraser.
`pick_place`	100	73,308	61.1	0.94 m	Pick up the object and place it at the target location.
`pick_usb`	102	68,670	57.2	0.75 m	Pick up the USB plug.
`stack_bowl`	100	47,539	39.6	0.71 m	Stack the bowl.
`insert_screw`	48	31,046	25.9	0.56 m	Insert the nut onto the screw.
`push_hongzhong_pos2`	34	28,559	23.8	0.63 m	Push the red dragon Mahjong tile to the target position.
`push_hongzhong_pos3`	25	25,628	21.4	0.82 m	Push the red dragon Mahjong tile to the target position.
`push_hongzhong_pos1`	33	18,356	15.3	0.49 m	Push the red dragon Mahjong tile to the target position.
`cut_banana`	38	10,512	8.8	0.20 m	Cut the banana with a knife.
`pick_hongzhong`	18	9,413	7.8	0.61 m	Pick up the red dragon Mahjong tile.
`insert_usb`	2	601	0.5	0.23 m	Insert the USB plug into the port.

Episodes carry the fields task, episode_index, instruction, num_frames, duration, fps, end_effector_path_m, end_effector_z_min_m, end_effector_z_max_m, gripper_min, gripper_max and tactile_sensor.

The three push_hongzhong tasks are the same push from three different start positions, and insert_usb holds two trajectories against pick_usb's 102, so the set is not balanced across tasks.

Notes on the conversion

The source ships one Zarr v2 group per task in the Open-X-Tactile layout, with every trajectory laid end to end and meta/episode_ends marking the boundaries. The per-frame clock restarts at each boundary, so episodes are cut there and each one starts at zero.

The four image streams arrive as raw uint8 arrays and are encoded to Annex-B H.264 without B-frames, one access unit per message.

The end-effector rotation is published as a rotation vector in radians and is carried as one, in /end-effector.plot, alongside the quaternion /end-effector-pose needs.

Arm joint angles are not in the source and are not carried. The gripper is a single opening scalar.

The release excludes 21 trajectories from a no_tactile batch and 4 that failed source validation, leaving the 703 here. Episode and frame counts match the release's own source_qc_summary.json on all 12 tasks.

License & attribution

The source release is distributed under CC BY 4.0, and this conversion is distributed under the same license.

bibtex

@article{lou2026dreamtac,
  title   = {Dream-Tac: A Unified Tactile World Action Model for
             Contact-Rich Robot Manipulation},
  author  = {Lou, Yunfan and Ye, Yifan and Fu, Yankai and Cen, Jun and
             Chi, Xiaowei and Lyu, Yaoxu},
  journal = {arXiv preprint arXiv:2606.08737},
  year    = {2026}
}

Changes from the source: conversion to the FiftyOne MCAP flavor, encoding of the four image streams to H.264, splitting of the concatenated task groups at their episode boundaries, and encoding of the pose, gripper and instruction streams as message streams.