CoolFace
Modelpublic

carloshuang1224/mega-asr-mlx

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face
0likes5downloads
Model Card

Mega-ASR MLX

A high-accuracy automatic speech recognition model optimized for Apple Silicon (M1/M2/M3/M4) using MLX.

This model is an MLX port of Qwen3-ASR-1.7B with Mega-ASR LoRA adapters pre-merged, plus an Audio Quality Router for degraded audio detection.

Model Architecture

ComponentDetails
Audio Encoder3-layer Conv2D stem + 24-layer Transformer (d_model=1024, 16 heads)
DecoderQwen3 1.7B — 28 layers, hidden=2048, 16 GQA heads (8 KV)
Alignerconv_out (7680→1024) + proj1 (1024→1024) + proj2 (1024→2048)
RouterMini Transformer (d_model=256, 4 layers, 4 heads) — binary audio quality classifier

Total parameters: ~1.7B (decoder) + ~350M (encoder)

Quick Start

Install

bash
pip install mega-asr-mlx

Download the model

bash
huggingface-cli download carloshuang1224/mega-asr-mlx --local-dir ./mega-asr-mlx

Inference

CLI:

bash
mega-asr --audio speech.wav --language English

Python API:

python
from mega_asr_mlx import MegaASRMLX

model = MegaASRMLX("./mega-asr-mlx")
text = model.transcribe("speech.wav", language="English")
print(text)

Advanced Usage

python
from mega_asr_mlx import MegaASRMLX

model = MegaASRMLX(
    "./mega-asr-mlx",
    use_lora=True,           # always use LoRA (pre-merged)
    router_threshold=0.5,    # audio quality routing threshold
    max_new_tokens=256,      # max generated tokens
)

# Transcribe with routing info
result = model.transcribe("speech.wav", return_route_info=True)
# {"text": "...", "use_lora": True, "degraded_prob": 0.12}

# Transcribe numpy array
import soundfile as sf
audio, sr = sf.read("speech.wav")
text = model.transcribe(audio, sr=sr)

Performance

On Apple Silicon (M-series), with KV cache optimization:

  • —~50 tokens/second generation speed
  • —Decoder weights: 3.8 GB (float16)
  • —Encoder weights: 606 MB (float16)
  • —Router weights: 2.3 MB (float16)

Files

mega-asr-mlx/
├── decoder.safetensors      # Qwen3 decoder weights (3.8 GB)
├── encoder.safetensors      # Audio encoder weights (606 MB)
├── router.safetensors       # Audio quality router (2.3 MB)
├── config.json              # Model configuration
├── decoder_config.json      # Decoder architecture config
├── router_config.json       # Router architecture config
├── preprocessor_config.json # Audio preprocessing (Whisper-style mel)
├── generation_config.json   # Generation defaults
├── tokenizer_config.json    # Qwen2 tokenizer config
├── vocab.json               # Token vocabulary
├── merges.txt               # BPE merges
└── chat_template.json       # Qwen3-ASR chat template

Source Code

The full inference pipeline is bundled in the mega-asr-mlx pip package. After installing it, the source is available in the mega_asr_mlx/ directory.

Key files:

  • —src/inference.py — End-to-end transcription script
  • —src/model.py — MegaASRMLX class with full pipeline
  • —src/audio_encoder.py — MLX audio encoder implementation
  • —src/router.py — Audio quality router
  • —src/convert.py — Weight conversion from PyTorch to MLX

How It Works

  1. 1.Audio Input → 16kHz mono waveform
  2. 2.Mel Extraction → 128-bin log-mel spectrogram (Whisper-style)
  3. 3.Audio Encoder → Conv2D stem downsamples 8×, then 24 Transformer layers encode
  4. 4.Decoder Input → Template: <|im_start|>user\n<|audio_start|> [encoder_outputs] <|audio_end|><|im_end|>\n<|im_start|>assistant\n
  5. 5.Generation → Qwen3 decoder auto-regressively generates text with KV cache

Credits

  • —Base model: Qwen/Qwen3-ASR-1.7B by Alibaba Qwen Team
  • —LoRA adapter: Mega-ASR by VoiceInk
  • —MLX port: Converted and optimized for Apple Silicon

License

Apache 2.0