zzzrw/GEM-250K

<div align="center"> <h1>GEM: Generative Supervision Helps Embodied Intelligence</h1>

<p align="center"> <a href="https://zhaorw02.github.io/">Ruowen Zhao</a>¹, Bangguo Li¹, <a href="https://liuzuyan.github.io/">Zuyan Liu</a>^1,2,†, Yinan Liang¹, <a href="https://jamesyjl.github.io/">Junliang Ye</a>¹, <a href="https://liuff19.github.io/">Fangfu Liu</a>¹, <br> Diankun Wu¹, <a href="https://thuwzy.github.io/">Zhengyi Wang</a>¹, <a href="https://yuxumin.github.io/">Xumin Yu</a>², <a href="https://raoyongming.github.io/">Yongming Rao</a>^2,✉, <a href="https://ancientmooner.github.io/">Han Hu</a>², <a href="https://ml.cs.tsinghua.edu.cn/~jun/index.shtml">Jun Zhu</a>^1,✉ <br> ^†Project Lead.^✉Corresponding Author. <br> ¹Tsinghua University, ²Tencent Hunyuan </p>

<a href='https://zhaorw02.github.io/GEM/'><img src="https://img.shields.io/badge/Project-Page-Green" alt="Project Page"></a> <a href="https://arxiv.org/abs/2605.28548"><img src="https://img.shields.io/badge/Paper-Arxiv-red?logo=arxiv" alt="Paper"></a> <a href="https://github.com/zhaorw02/GEM/"><img src="https://img.shields.io/badge/GitHub-Repo-181717?logo=github&logoColor=white" alt="GitHub"></a> <a href="https://huggingface.co/zzzrw/GEM-2B/"><img src="https://img.shields.io/badge/Models-HuggingFace-yellow?logo=huggingface" alt="Models"></a> <a href="https://huggingface.co/datasets/zzzrw/GEM-250K/"><img src="https://img.shields.io/badge/Dataset-HuggingFace-yellow?logo=huggingface" alt="Dataset"></a>

</div>

<div align="center"> <video src="https://huggingface.co/datasets/zzzrw/GEM-250K/resolve/main/assets/GEM-demo.mp4" controls autoplay muted loop width="85%"></video> </div>

Abstract

Embodied Vision-Language Models (VLMs) have demonstrated impressive performance and generalization in robotics, particularly within Vision-Language-Action frameworks. However, a significant gap remains between the high-level semantic focus of standard text-guided pre-training paradigms and the low-level spatial and physical knowledge critical for execution in embodied environments. In this paper, we introduce GEM, a Generative-supervised Embodied vision-language Model designed to bridge this divide. We propose integrating a depth map generation task directly into the VLM pre-training phase. By training this generative objective jointly with the main model, we observe substantial improvements in embodied intelligence, significantly enhancing both semantic understanding and physical operation capabilities. To support this paradigm, we curate and release GEM-4M, a comprehensive large-scale dataset featuring a mixture of grounding, reasoning, and planning data paired with high-quality depth supervision. Extensive experiments demonstrate that GEM achieves state-of-the-art results across diverse embodied benchmarks. Furthermore, our deployed action model, GEM-VLA, exhibits vastly superior task execution abilities in both simulation environments and real-world evaluations.

<div align="center"> <img src="assets/overview.png" alt="GEM Teaser" width="85%"> </div>

Dataset Structure

GEM-250K/
├── assets/
│   ├── GEM-demo.mp4
│   ├── overview.png
├── data/
│   ├── embodied_grounding-00000-of-00002.parquet
│   ├── embodied_grounding-00001-of-00002.parquet
│   ├── spatial_reasoning-00000-of-00001.parquet
│   └── spatiotemporal_planning-00000-of-00001.parquet
└── README.md

Citation

If you find this dataset helpful, please cite:

@article{zhao2026gem,
  title={GEM: Generative Supervision Helps Embodied Intelligence},
  author={Zhao, Ruowen and Li, Bangguo and Liu, Zuyan and Liang, Yinan and Ye, Junliang and Liu, Fangfu and Wu, Diankun and Wang, Zhengyi and Yu, Xumin and Rao, Yongming and others},
  journal={arXiv preprint arXiv:2605.28548},
  year={2026}
}