CoolFace
Datasetpublic

microsoft/MuseVLA-dataset

MuseVLA Dataset Multi-modal robot manipulation dataset with synchronized RGB, depth, acoustic, thermal, and radar streams. Released as two parts (dataset_01/, dataset_02/) sharing the same per-episode layout. Together they cover ~1400 episodes across 11 instructions (towel / clothes / box / item / drink manipulation). Per-episode contents {episode_name}/ ├── video.mp4 # RGB, 1280×720, 30 fps ├── mask/video.mp4 #… See the full description on the dataset page: https://huggingface.co/datasets/microsoft/MuseVLA-dataset.

sourceHugging Facecc-by-nc-4.0updated 1mo agoView on Hugging Face
2likes453downloads
Dataset Card

<div align="center"> <span style="font-size:32px;">MuseVLA Dataset</span> </div>

<p align="center"> <a href="https://arxiv.org/abs/2606.17598"><img src="https://img.shields.io/badge/arXiv-Paper-red?logo=arxiv&logoColor=white" alt="arXiv"></a> <a href="https://github.com/microsoft/MuseVLA"><img src="https://img.shields.io/badge/Code-GitHub-181717?logo=github&logoColor=white" alt="Code Repository"></a> <a href="https://huggingface.co/microsoft/MuseVLA"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-blue" alt="Hugging Face Model"></a> </p>

Multi-modal robot manipulation dataset with synchronized RGB, depth, acoustic, thermal, and radar streams. Released as two parts (dataset_01/, dataset_02/) sharing the same per-episode layout. Together they cover ~1400 episodes across 11 instructions (towel / clothes / box / item / drink manipulation).

Per-episode contents

{episode_name}/
├── video.mp4                           # RGB, 1280×720, 30 fps
├── mask/video.mp4                      # SAM3 segmentation mask
├── depth/images/                       # depth jpgs
├── acoustic/{images,images_masked}/    # spectrogram + 3 overlay variants
├── radar/{images,images_masked}/       # radar + 3 overlay variants
├── thermal/{left,right,images,images_masked}/   # left / right / merged / overlays
├── 6d_pose/right_arm.npy               # end-effector trajectory
├── arm_command/, arm_status/, ee_command/, hand_status/   # control signals

Annotations live under *_labels/{episode}.json with per-segment instruction, start, end, plus _validity.json recording SAM3 validity. Top-level instructions_all.json, valid_samples.json, and statistics_6dpose_delta_NRt0.json are provided for indexing and normalization.

Statistics

EpisodesValidAnnotated segmentsUnique instructionsRGBFrame rate
1397 (1279 + 118)1010 (913 + 97)1930 (1737 + 193)111280 × 72030 fps

Citation

bibtex
@misc{liu2026musevlaadaptivemultimodalsensing,
  title={MuseVLA: An Adaptive Multimodal Sensing Vision-Language-Action Model for Robotic Manipulation},
  author={Xingyuming Liu and Ruichun Ma and Heyu Guo and Qixiu Li and Qingwen Yang and Lin Luo and Shiqi Jiang and Chenren Xu and Jiaolong Yang and Baining Guo},
  year={2026},
  eprint={2606.17598},
  archivePrefix={arXiv},
  primaryClass={cs.RO},
  url={https://arxiv.org/abs/2606.17598},
}

License

CC-BY-NC-4.0 (research use only).

microsoft/MuseVLA-dataset · CoolFace