CoolFace
Modelpublic

giangndm/Qwen3-ASR-1.7B-encoder

sourceHugging Faceapache-2.0updated 22d agoView on Hugging Face
0likes55downloads
Model Card

Qwen3-ASR-1.7B Audio Encoder & Projector (bfloat16 Transformers SafeTensors)

This repository provides the standalone bfloat16 `safetensors` Audio Encoder & Multi-Modal Projector extracted from `Qwen/Qwen3-ASR-1.7B-hf`.

It can be loaded directly with transformers.AutoModel via trust_remote_code=True without requiring the full 1.7B language model backbone.

📐 Architecture & Specifications

ParameterValueDescription
Audio Tower Architecture24 Transformer Layers (16 heads, FFN 4096)Whisper/Conformer-style Transformer Encoder
Audio Hidden Dimension1024-dimRaw output of the Audio Tower (hidden_states)
Projected Dimension2048-dimOutput after Multi-Modal Projector matching Qwen3 LLM (last_hidden_state)
Frame Rate12.5 frames/s (80ms / frame)$8\times$ temporal downsampling from 10ms log-mel frames
Audio FrontEnd128 Mel Bins, 16 kHz3 Conv2D downsampling stages ($2\times 2\times 2 = 8\times$)
Weight Format & Precisionmodel.safetensors (`bfloat16`)Lightweight standalone package: ~605 MB

💻 Quick Start with Hugging Face transformers

python
import torch
import torchaudio
from transformers import AutoModel

# 1. Load Audio Encoder from Hugging Face
model_id = "giangndm/Qwen3-ASR-1.7B-encoder"
model = AutoModel.from_pretrained(
    model_id, 
    trust_remote_code=True, 
    torch_dtype=torch.bfloat16
).to("cuda")
model.eval()

# 2. Load Any 16kHz Audio File
waveform, sr = torchaudio.load("path/to/audio.wav") # [1, T_samples]

# 3. Extract 128-dim Log Mel-Fbank Features (Using the Built-in Helper)
fbank = model.extract_fbank(waveform.to("cuda"), sample_rate=sr).to(torch.bfloat16) # [1, 1, T_mel, 128]

# 4. Forward Pass through Audio Tower + Projector
with torch.no_grad():
    output = model(fbank, return_projected=True)
    audio_embeds = output.last_hidden_state # [1, T_frames, 2048] (12.5 Hz / 80ms rate)

print("Audio Embeddings Shape:", audio_embeds.shape)
# Example: 1-second audio -> torch.Size([1, 12, 2048])

📦 Internal Layer Structure:

  • —`conv2d1` $\to$ `conv2d2` $\to$ `conv2d3`: 3 Conv2D blocks ($3\times 3$, stride 2) performing $8\times$ downsampling on mel bins and time.
  • —`conv_out`: Linear projection layer mapping $7680 \to 1024$.
  • —`layers.0` ... `layers.23`: 24 Transformer self-attention layers with Pre-LayerNorm.
  • —`ln_post`: Final LayerNorm of the Audio Tower.
  • —`projector_linear1` $\to$ `GELU` $\to$ `projector_linear2`: 2-layer MLP projector mapping $1024 \to 2048$ into the Text LLM embedding space.