kandinskylab/KVAE-Audio
<div align="center"> <picture> <img src="assets/kvae_audio.png"> </picture>
<a href="https://github.com/kandinskylab/kvae-audio">GitHub</a> | <a href="https://habr.com/ru/companies/sberbank/articles/1053410/">Habr article</a> | <a href="https://kandinskylab.ai/">Project Page</a> | <a href="https://huggingface.co/papers/2608.05798">Technical Report</a> </div>
<h1>KVAE-Audio</h1>
KVAE-Audio is a continuous, full-band (48 kHz) audio autoencoder. It compresses raw waveforms into compact continuous latents and reconstructs them with high fidelity across speech, music, and general sound. The model is designed not only for faithful reconstruction, but as a latent space for generative models — in our internal text-to-audio pipeline, swapping the autoencoder for KVAE-Audio improves generation quality under a fixed generator.
Evaluation results
Evaluation of latent space qualities for generation
Generative quality is established under a fixed generator — same DiT architecture, training data, and number of steps — varying only the autoencoder. We report objective generation metrics and blind human side-by-side below.
<img src="assets/sbssamel.png" />
<img src="assets/sbs_moviegen.png" />
<img src="assets/sbs_mmaudio.png" />
AudioCaps test set
Song Describer
LibriSpeech test-clean
Reconstructions
Reconstruction is evaluated on open datasets across domains (the released weights directly substantiate these numbers). Baselines: [MMAudio 44.1 kHz](https://arxiv.org/abs/2412.15322) VAE, [DACVAE from MovieGen Audio](https://arxiv.org/abs/2410.13720), [SAME-L](https://arxiv.org/abs/2605.18613) (Stable Audio 3 VAE).