JacobLinCool/TEA-ASR-1

1---2base_model: Qwen/Qwen3-ASR-1.7B3license: mit4library_name: transformers5pipeline_tag: automatic-speech-recognition6language:7  - zh8  - en9tags:10  - automatic-speech-recognition11  - taiwan-mandarin12  - traditional-chinese13  - code-switching14  - qwen3-asr15  - speech16---17 18# TEA-ASR-1 · Taiwan Everyday Audio 🍵19 20**TEA-ASR is an open, drop-in speech-recognition model purpose-built for Taiwan Mandarin.** It turns real speech21into natural **Traditional Chinese** with authentic **Taiwan vocabulary**, and it22stays robust through the everyday **Mandarin–English code-switching** common in Taiwan. Adapted from the23state-of-the-art **Qwen3-ASR** foundation and merged into a single self-contained checkpoint, TEA-ASR **loads and24runs exactly like stock Qwen3-ASR** — no converters, no post-processing — while matching or surpassing both a25dedicated Taiwan specialist and a large multilingual model on every public benchmark we evaluate.26 27`TEA-ASR-1` is the **2B flagship (best accuracy)**.28A companion **TEA-ASR-1-mini** shares the identical recipe — see [`JacobLinCool/TEA-ASR-1-mini`](https://huggingface.co/JacobLinCool/TEA-ASR-1-mini).29 30## Key features31 32- 🎯 **Built for Taiwan Mandarin** — Traditional script **and** Taiwan-style word choice, produced by the model33  itself.34- 🔀 **Code-switch robust** — handles natural zh-en mixing instead of translating Mandarin into English.35- 🧩 **Drop-in Qwen3-ASR compatible** — same loading and inference API as the base model; nothing else to install36  or call.37- 🪶 **Lightweight adaptation** — a small decoder LoRA on a frozen audio encoder, trained on a few hours of public38  audio, then merged for deployment.39 40## Quick start41 42```bash43pip install qwen-asr44```45 46```python47from qwen_asr import Qwen3ASRModel48 49model = Qwen3ASRModel.from_pretrained("JacobLinCool/TEA-ASR-1")50result = model.transcribe(audio="utterance.wav", language="Chinese")[0]51print(result.text)   # -> Traditional Chinese with Taiwan lexicon52```53 54Set `language="Chinese"` for Taiwan speech (recommended). You can also pass a `context=` string of hotwords55(names, jargon) for contextual biasing, exactly as with the base Qwen3-ASR.56 57## Benchmark results58 59Mixed Error Rate (MER%, **lower is better**), all numbers from a **single self-measured run under one protocol**60(see [Evaluation](#evaluation)). Columns: the two TEA-ASR models, the original (unadapted) **Qwen3-ASR** bases, and61two references — **Breeze-ASR-25** (a Taiwan-specialist ASR) and **Whisper-large-v3**. **Bold = this model.**62 63| Benchmark | TEA-ASR-1 | TEA-ASR-1-mini | Qwen3-ASR-1.7B | Qwen3-ASR-0.6B | Breeze-ASR-25 | Whisper-large-v3 |64|---|---|---|---|---|---|---|65| CommonVoice 19 (zh-TW) | **3.64** | 5.14 | 3.90 | 5.79 | 8.03 | 10.17 |66| ASCEND (zh-en) | **10.59** | 12.49 | 10.57 | 12.54 | 17.53 | 19.61 |67| CSZS (zh-en) | **10.98** | 13.21 | 11.03 | 16.03 | 12.18 | 23.24 |68| NTUML2021 | **6.80** | 7.37 | 10.12 | 11.03 | 7.50 | 9.68 |69 70**How to read this.** **TEA-ASR-1** is the flagship model on this page.71Across the suite, **TEA-ASR-1 posts the best (or tied-best) error rate on every benchmark**, ahead of the72Taiwan-specialist Breeze-ASR-25 and far ahead of Whisper-large-v3; **TEA-ASR-1-mini** delivers most of that quality73at well under half the parameters (780M vs 2B). Against the unadapted **Qwen3-ASR** base, the gain in this content-folded74recognition metric is largest on in-domain lectures (NTUML2021); on the other sets recognition is on par or75better — and, importantly, the metric **folds away script differences** (see Evaluation), so it does *not* reflect76the decisive practical change: TEA-ASR emits **Traditional script and Taiwan vocabulary natively**, whereas the77base produces Simplified script.78 79## Speed & memory80 81Measured on **NVIDIA RTX 5090 (32 GB)** (bf16, batch 1, 50 utterances, greedy decode). **xRT = audio seconds processed per82wall-clock second** (higher is faster); **RTF = wall-clock / audio** (lower is faster); **peak VRAM** is the maximum83allocated during inference.84 85| Model | Params | xRT ↑ | RTF ↓ | Peak VRAM (GB) ↓ |86|---|---|---|---|---|87| TEA-ASR-1 | 2B | 11.0 | 0.091 | 4.16 |88| TEA-ASR-1-mini | 780M | 8.1 | 0.124 | 1.65 |89| Breeze-ASR-25 | 1.54B | 5.5 | 0.182 | 4.41 |90| Whisper-large-v3 | 1.54B | 4.7 | 0.214 | 4.41 |91 92## Figures93 94**Accuracy across the four public benchmarks** (content-fold MER%, lower is better):95 96![Accuracy across benchmarks](bench_mer.png)97 98**Speed and memory** (single GPU, bf16, batch 1):99 100![Speed and memory](bench_speed_vram.png)101 102**Ablation — tokenizer × finetune.** Content MER isolates the *finetune* gain (the script fold hides tokenizer effects); raw MER isolates the *tokenizer-first* localization that makes the output Traditional + Taiwan-lexicon:103 104![Ablation](ablation_2x2.png)105 106## Evaluation107 108- **Metric — Mixed Error Rate (MER).** Character Error Rate for Chinese and Word Error Rate for the English tokens,109  computed jointly per utterance and micro-averaged.110- **Content fold (applied uniformly to every dataset and every system).** Before scoring, both the reference and111  the hypothesis are normalized to a common form — **converted to Simplified Chinese with OpenCC (`t2s`)**,112  lowercased, and stripped of punctuation. This isolates *recognition* from *script style*, so a Simplified-output113  model (e.g. the base) and a Traditional-output model (TEA-ASR) are compared fairly on content. (TEA-ASR's actual114  output is Traditional; the fold is only for scoring.)115- **Decoding.** TEA-ASR and Qwen3-ASR are decoded with `language=Chinese`; Whisper-large-v3 and Breeze-ASR-25 use116  their own automatic language detection. All systems are scored with the **same code on the same public splits**;117  we do not import numbers reported elsewhere.118 119| Dataset | What it tests | Eval split (n) |120|---|---|---|121| **CommonVoice 19 (zh-TW)** | Read Taiwan-Mandarin speech | full test (5013) |122| **ASCEND** | Spontaneous Mandarin–English code-switch conversation | full test (1315) |123| **CSZS (zh-en)** | Zero-resource code-switch benchmark | full test (3176) |124| **NTUML2021** | Mandarin lecture speech (university ML course) | test[:2000] |125 126- **No train/test leakage.** Fine-tuning used **only** the *training* pools, disjoint from every evaluation127  split: the NTUML2021 *train* split, the ASCEND *train* split, and a CommonVoice slice drawn from128  `validated_without_test` (CommonVoice's official non-test pool, disjoint from its *test* split). Evaluation129  therefore runs on the **full, untouched** CommonVoice / ASCEND / NTUML2021 *test* splits; CSZS is a separate130  dataset not used in training at all. Every number above is leak-free.131 132## How it was built133 134- **Base** `Qwen/Qwen3-ASR-1.7B` (frozen AuT audio encoder + Qwen3 decoder).135- **Adaptation**: a rank-16 **decoder-only LoRA** trained on **a few hours of public audio** (CommonVoice zh-TW,136  ASCEND, NTUML2021), with general + code-switch **replay** to preserve the base model's broad and bilingual137  ability. The audio encoder is left frozen.138- **Localization**: Traditional-script + Taiwan-lexicon output is rendered through the model's **own tokenizer**139  (the surface mapping is baked once at build time); there is **no post-processing at inference** — the140  Traditional output comes straight from the model's own tokenizer decode.141- **Packaging**: the adapter is **merged** into the base and the localized tokenizer is shipped with it, so the142  release is a single drop-in checkpoint that loads like stock Qwen3-ASR.143- **Decoding tip**: pass `language="Chinese"` for Taiwan speech; this also prevents translation-style outputs on144  dense code-switch.145 146## Limitations147 148- **Dense synthetic code-switch (CSZS)**: the smaller TEA-ASR-1-mini trails the Taiwan specialist on this set; the149  flagship TEA-ASR-1 leads it. For heavy code-switch, prefer TEA-ASR-1.150- **Scope**: validated on the Qwen3-ASR family (0.6B and 1.7B); the released models load via the `qwen-asr` package,151  exactly like the base.152 153## Citation154 155```bibtex156@misc{teaasr2026,157  title  = {Tokenizer-First Adaptation of Mandarin ASR to Taiwan Mandarin},158  author = {TEA-ASR contributors},159  year   = {2026},160  note   = {TEA-ASR (Taiwan Everyday Audio); adapted from Qwen3-ASR}161}162```163 164Built on [Qwen3-ASR](https://huggingface.co/Qwen/Qwen3-ASR-1.7B) (Apache-2.0). The TEA-ASR adaptation and this checkpoint are165released under the **MIT License**; the underlying Qwen3-ASR weights remain subject to the Apache-2.0 license and its166attribution/NOTICE terms.167