nyu-visionx/siglip2_decoder

Scaling Text-to-Image Diffusion Transformers with Representation Autoencoders (Scale-RAE)

This repository contains artifacts related to the paper Scaling Text-to-Image Diffusion Transformers with Representation Autoencoders.

• —Project Page: https://rae-dit.github.io/scale-rae/
• —GitHub Repository: https://github.com/ZitengWangNYU/Scale-RAE

Introduction

Representation Autoencoders (RAEs) provide a simplified and powerful alternative to VAEs for large-scale text-to-image generation. Scale-RAE demonstrates that training diffusion models in high-dimensional semantic latent spaces (using encoders like SigLIP-2) leads to faster convergence, better generation quality, and improved stability compared to state-of-the-art VAE-based foundations.

Usage

For detailed instructions on installation, training, and inference, please visit the official GitHub repository.

This decoder is also directly compatitable with original RAE [codebase](https://github.com/bytetriper/RAE). Try it out by simply swapping the encoder with google/siglip2-so400m-patch14-224!

Citation

@article{scale-rae-2026,
  title={Scaling Text-to-Image Diffusion Transformers with Representation Autoencoders},
  author={Shengbang Tong and Boyang Zheng and Ziteng Wang and Bingda Tang and Nanye Ma and Ellis Brown and Jihan Yang and Rob Fergus and Yann LeCun and Saining Xie},
  journal={arXiv preprint arXiv:2601.16208},
  year={2026}
}