boardd/dexwild-dataset

<p align="center"> <img src="website_assets/imgs/title.png" width="100%"> </p>

<div align="center">

Tony Tao^\, [Mohan Kumar Srirama](https://www.mohansrirama.com/)^\, Jason Jingzhou Liu, Kenneth Shaw, Deepak Pathak

Robotics: Science and Systems (RSS) 2025

[[Project page]](https://dexwild.github.io/) [[Video]](https://youtu.be/oMaamSkcl5E) [[ArXiv]](https://arxiv.org/abs/2505.07813)

[[Main Code Repo]](https://github.com/dexwild/dexwild) [[Policy Training]](https://github.com/dexwild/dexwild-training)

[[Hardware Guide]](https://resisted-salad-9e6.notion.site/DexWild-Hardware-Setup-Guide-20eee3f68d27801b8eb8dfde3a5bb7c4?source=copylink) [[Data Collection Guide]](https://resisted-salad-9e6.notion.site/DexWild-Data-Collection-Guide-20fee3f68d27803b9a88ef9847e292d4?source=copylink)

![License: MIT]()

</div>

Data Breakdown

The DexWild dataset is one of the largest, high quality human dexterous hand datasets with a huge diversity of objects and environments. In total, we collect 9,505 Episodes across 5 tasks and 93 environments. 33+ Hours equating to 3.5M+ Transitions!

Florist Human - 1,030 Episodes, 310,571 Transitions Robot - 248 Episodes, 92,954 Transitions

Clothes Folding Human - 1,123 Episodes, 294,367 Transitions Robot - 295 Episodes, 123,385 Transitions

Spray Human - 2,820 Episodes, 886,460 Transitions Robot - 388 Episodes, 162,862 Transitions

Toy Cleanup Human - 2,323 Episodes, 1,251,649 Transitions Robot - 546 Episodes, 316,289 Transitions

Pouring Human - 621 Episodes, 181,541 Transitions Robot - 111 Episodes, 30,880 Transitions

Data Structure

Data is saved in HDF5 Format in the following general structure. Note that single arm tasks are missing a few of these folders.

data
├── ep_0000
│   ├── intergripper
│   │   └── intergripper.pkl
│   ├── left_leapv2
│   │   └── left_leapv2.pkl
│   ├── left_manus
│   │   └── left_manus_full.pkl, left_manus.pkl
│   ├── left_pinky_cam
│   │   └── timestamp1.jpg, timestamp2.jpg, ...
│   ├── left_thumb_cam
│   │   └── timestamp1.jpg, timestamp2.jpg, ...
│   ├── left_tracker
│   │   └── left_tracker_cam_frame_abs.pkl, ...
│   ├── right_leapv2
│   │   └── right_leapv2.pkl
│   ├── right_manus
│   │   └── right_manus_full.pkl, right_manus.pkl
│   ├── right_pinky_cam
│   │   └── timestamp1.jpg, timestamp2.jpg, ...
│   ├── right_thumb_cam
│   │   └── timestamp1.jpg, timestamp2.jpg, ...
│   ├── right_tracker
│   │   └── right_tracker_cam_frame_abs.pkl, ...
│   ├── timesteps
│   │   └── timesteps.txt
│   ├── zed
│   │   └── zed_ts.pkl  
│   └── zed_obs
│       └── timestamp1.jpg, timestamp2.jpg, ...
├── ep_0001
├── ep_0002
├── ...

Data Recombination

Since HuggingFace has a limit on single file size, you must recombine the data to obtain the full dataset for certain HDF5 files.

After downloading the split files, follow this guide.

Merging Instructions:

1. 1.Navigate to the Download Directory:

   cd path_to_downloaded_files

1. 1.Merge Files Using `cat`:

Use the cat command to concatenate the split parts. Replace filename.part_* with your actual file name.

   cat filename.part_* > filename.tar

Example:

   cat DexWild_Data.part_* > DexWild_Data.tar

1. 1.Verify the Merged File:

Ensure that the merged file size matches the original file size before merging. You can use the ls -lh command to check file sizes.

   ls -lh DexWild_Data.tar

1. 1.Extract the Dataset:

Once merged, extract the dataset using the tar command:

   tar -xvf DexWild_Data.tar

Citation

If you find this project useful, please cite our work:

@article{tao2025dexwild,
      title={DexWild: Dexterous Human Interactions for In-the-Wild Robot Policies},
      author={Tao, Tony and Srirama, Mohan Kumar and Liu, Jason Jingzhou and Shaw, Kenneth and Pathak, Deepak},
      journal={Robotics: Science and Systems (RSS)},
      year={2025}}