Modelpublic

FlexiSLM/Qwen2_5-Omni-Audio_Encoder

sourceHugging Faceapache-2.0updated 1mo agoView on Hugging Face

0likes35downloads

Model Card

Qwen2.5-Omni Audio Encoder

Standalone audio encoder extracted from Qwen/Qwen2.5-Omni-7B for FlexiSLM.

This checkpoint keeps only Qwen2_5OmniAudioEncoder (thinker.audio_tower.*: conv layers, 32 transformer layers, ln_post, and proj). The Omni thinker LLM, talker, vision tower, and tokenizer are omitted.

Property	Value
Parameters	640M
Size	~1.19 GiB (`bfloat16`)
Output dim	3584
Mel bins	128

—Paper: arXiv:2606.31247
—FlexiSLM code: AmphionTeam/FlexiSLM

Usage with FlexiSLM

Required for both 7B and 0.5B FlexiSLM inference and training. Shared by all Stage 1 / Stage 2 / Stage 3 recipes.

Download

bash

MODEL_ROOT="$PWD/models"
hf download FlexiSLM/Qwen2_5-Omni-Audio_Encoder \
  --local-dir "$MODEL_ROOT/Qwen2_5-Omni-Audio_Encoder"

With FlexiSLM auto_download=True, this encoder is fetched automatically together with the Stage 2 SLM, SenseVoice, and FlexiCodec assets.

Inference config (Python)

python

from pathlib import Path
from src.inference_flexislm import FlexiSLMInferenceConfig

model_root = Path.cwd() / "models"
config = FlexiSLMInferenceConfig(
    checkpoint="stage2_7B",  # or "stage2_0.5B"
    model_path=str(model_root / "FlexiSLM-7B-Stage2"),
    qwen25o_encoder_path=str(model_root / "Qwen2_5-Omni-Audio_Encoder"),
    qwen25o_encoder_config_path=str(
        model_root / "Qwen2_5-Omni-Audio_Encoder/config.json"
    ),
    # ... FlexiCodec / SenseVoice / flow-matching paths ...
)

Training / YAML

yaml

use_qwen25omni_feature: true
qwen25omni_encoder_path: models/Qwen2_5-Omni-Audio_Encoder
qwen25omni_encoder_config_path: models/Qwen2_5-Omni-Audio_Encoder/config.json

Full TTS / ASR / audio QA / speech-to-speech examples: FlexiSLM Inference Guide.

Load with Transformers

Weights are loadable with transformers.models.qwen2_5_omni.modeling_qwen2_5_omni.Qwen2_5OmniAudioEncoder.

License

Citation

If you use this encoder as part of FlexiSLM:

bibtex

@misc{li2026flexislmdynamiccontrollableframe,
      title={FlexiSLM: A Dynamic and Controllable Frame Rate Spoken Language Model},
      author={Jiaqi Li and Chaoren Wang and Xiaohai Tian and Mingjie Chen and Xinyu Liang and Xu Li and Yufan Lin and Junwen Qiu and Jun Zhang and Lu Lu and Haizhou Li and Zhizheng Wu},
      year={2026},
      eprint={2606.31247},
      archivePrefix={arXiv},
      primaryClass={cs.SD},
      url={https://arxiv.org/abs/2606.31247},
}