facebook/dexwm

<!-- Data from DexWM: World Models for Learning Dexterous Hand-Object Interactions from Human Videos. 4 hours of exploratory sequences of random arm movements collected in RoboCasa. -->

<div align="center">

<h1><strong>DexWM: World Models for Learning Dexterous Hand-Object Interactions from Human Videos</strong></h1>

📄 Paper &nbsp;&nbsp;|&nbsp;&nbsp; 💻 Code &nbsp;&nbsp;|&nbsp;&nbsp; 🌐 Project Page

</div>

Description

This dataset contains the RoboCasa simulation data used in DexWM: World Models for Learning Dexterous Hand-Object Interactions from Human Videos. It includes two data regimes for training and evaluation of DexWM.

• —RoboCasa Random: Contains exploratory_movement and gripper_open_and_close sequences. These are random interaction trajectories collected using a Franka arm with an Allegro hand, used for model fine-tuning.
• —Pick-and-Place: Contains the pick-and-place-2.0 dataset, used exclusively for evaluating manipulation performance.

All data is stored in .hdf5 format, where each file contains sequential robot interaction trajectories, including states and actions for dexterous manipulation.

Citation

@article{goswami2025dexwm,
  title={World Models for Learning Dexterous Hand-Object Interactions from Human Videos},
  author={Goswami, Raktim Gautam and Bar, Amir and Fan, David and Yang, Tsung-Yen and Zhou, Gaoyue and Krishnamurthy, Prashanth and Rabbat, Michael and Khorrami, Farshad and LeCun, Yann},
  journal={arXiv preprint arXiv:2512.13644},
  year={2026}
}