BAAI-Humanoid/DECO-50

DECO: Decoupled Multimodal Diffusion Transformer for Bimanual Dexterous Manipulation with a Plugin Tactile Adapter

<p> <a href="https://arxiv.org/pdf/2602.05513"> <img src="https://img.shields.io/badge/arXiv-Paper-b31b1b?logo=arxiv"> </a> <a href="https://baai-humanoid.github.io/DECO-webpage/"> <img src="https://img.shields.io/badge/Project-Website-blue"> </a> <a href="https://github.com/BAAI-Humanoid/DECO"> <img src="https://img.shields.io/badge/GitHub-Code-black?logo=github"> </a> </p>

DECO-50 is a bimanual dexterous manipulation dataset with tactile sensing, comprising 50 hours of teleoperated data across 4 scenarios and 28 subtasks, totaling over 5 million frames collected on real dual-arm robots.

Dataset Structure

DECO-50/
├── task1/
│   ├── sub_task_1/
│   │   ├── episode_000000/
│   │   │   ├── colors/
│   │   │   │   ├── 000_color_0.jpg      # Left camera
│   │   │   │   ├── 000_color_1.jpg      # Right camera
│   │   │   │   └── ...
│   │   │   ├── tactiles/
│   │   │   │   ├── 000_left_ee_tactile.npy   # Left tactile sensor
│   │   │   │   ├── 000_right_ee_tactile.npy  # Right tactile sensor
│   │   │   │   └── ...
│   │   │   └── data.json   # original record file
│   │   │   └── data.pkl    # processed label file
│   │   ├── episode_000001/
│   │   └── ...
│   ├── sub_task_2/
│   └── ...
├── task2/
└── ...

Episode Label Format (data.pkl)

Using the Dataset

You can download the dataset and use it directly with our training scripts. Please note the following:

• —Task1–Task4 correspond to four manipulation scenarios: Pick and Place, Material Sorting, Waste Disposal, and Assembly. Each scenario is further divided into multiple subtasks.
• —Within each subtask, the file data.pkl uses condition_indx to distinguish different object instances and task types.
• —In our paper and released codebase, we train one model per scenario, resulting in four separate models in total. Alternatively, you may combine all scenarios into a single training set; however, in that case, you need to manually adjust condition_indx in data.pkl accordingly.
• —You may use the precomputed statistics provided with the DECO-50 dataset to normalize the observations and actions.

# Download the dataset
hf down "BAAI-Humanoid/DECO-50" --local-dir ./datasets
git clone https://github.com/BAAI-Humanoid/DECO
# Follow the instructions in https://github.com/BAAI-Humanoid/DECO?tab=readme-ov-file#2-prepare-for-dataset

Citation

If you use this dataset in your research, please cite:

@article{li2026deco,
  title={DECO: Decoupled Multimodal Diffusion Transformer for Bimanual Dexterous Manipulation with a Plugin Tactile Adapter},
  author={Xukun Li and Yu Sun and Lei Zhang and Bosheng Huang and Yibo Peng and Yuan Meng and Haojun Jiang and Shaoxuan Xie and Guacai Yao and Alois Knoll and Zhenshan Bing and Xinlong Wang and Zhenguo Sun},
  journal={arXiv preprint arXiv:2602.05513},
  year={2026}
}

License

This dataset is released under the Apache 2.0 license.