Nerdstorm/Qwen3-ASR-0.6B-Sinhala-8bit

Qwen3-ASR 0.6B Sinhala, 8-bit MLX

Qwen3-ASR 0.6B, fine-tuned to transcribe Sinhala in Sinhala script, with English words in English letters, and quantised to 8 bits for Apple silicon. It was trained on a Mac for LiveTranscribe; the training code and data preparation are in Nerdstorm/LiveTranscribe-Sinhala, and a blog series walks through how it was made.

It needs mlx-audio-swift [01dec7c](https://github.com/Blaizzy/mlx-audio-swift/commit/01dec7c9bdce3088a6b6b7ab9f2e403458195efb) or later, which computes Qwen3-ASR's audio features as Qwen does (#247). The model was trained on those features; earlier revisions compute different ones, and it transcribes worse with them.

Languages

It recognises the language itself, as the base model does. Sinhala is new; the other 30 languages are Qwen3-ASR's own. They're listed from the base model's strongest to its weakest, by its error rate on FLEURS in the Qwen3-ASR Technical Report (Table A.2b): the character error rate for character-based languages such as Chinese, Cantonese and Korean, and the word error rate for the rest. The report measured the full-precision base model. The "This model" column is measured as LiveTranscribe runs it, 8-bit, and with simpler text normalisation than the report's, so compare it with the 8-bit base model's figure beside it rather than with the report's.

Six languages were tested after fine-tuning: Sinhala on OpenSLR 52, and English, Chinese, Spanish, French and German on FLEURS (Results).

• —English's and Chinese's error rates rose by 6% or less, relative to the base model's.
• —Spanish, French and German rose by 19% to 33%, although all five were in the replay mix (How it was trained). The fine-tune splits more German compounds ("Wüsten sand" for "Wüstensand"), gets more French agreement wrong ("leur" for "leurs"), and mishears more names and rare words.
• —The other 25 languages had no replay and weren't tested, so they may have slipped as much or more.

The base model also recognises 22 Chinese dialects: Anhui, Dongbei, Fujian, Gansu, Guizhou, Hebei, Henan, Hubei, Hunan, Jiangxi, Ningxia, Shandong, Shaanxi, Shanxi, Sichuan, Tianjin, Yunnan, Zhejiang, Cantonese (Hong Kong accent), Cantonese (Guangdong accent), Wu and Minnan. None were tested here.

Results

Measured as LiveTranscribe runs it, with mlx-audio-swift 01dec7c: greedy decoding, no language given.

On a MacBook Pro with an M4 Pro, a Sinhala recording (4.3 s on average) took a median of 243 ms, 17 times faster than real time. An English one took 226 ms, 43 times faster. No output ran away or was cut short in any of these tests, and every FLEURS recording was recognised as its own language.

The test speakers' voices are new to the model, but most of their sentences aren't. OpenSLR 52's volunteers read from a shared pool of sentences, and 6,201 of the test recordings (71%) read one that someone in the training data also reads. On those it scores 6.06%. On the 2,547 with new sentences it scores 7.08%, which is the number to expect for dictation, where every sentence is new.

Sinhala word error rates look high on any model, and English words aren't the cause here: the recordings without any score about the same. One wrong letter makes the whole word wrong, and a word in this test averages about five characters, counting vowel signs, so a 7% character error rate spread evenly would already put a wrong character in about 30% of words. Of the words this model writes differently, 37% are one character off, often the same word with a different ending (කියල and කියලා, පස්සෙ and පස්සේ), and 14% of its word errors are only where a space goes. So the character error rate is the fairer measure. The base model is mlx-community/Qwen3-ASR-0.6B-8bit, run the same way.

How it was trained

• —Start: mlx-community/Qwen3-ASR-0.6B-8bit, the model LiveTranscribe ships, dequantised to float32.
• —Sinhala: OpenSLR 52, read speech from 478 speakers, split by speaker: 442 for training (172,134 utterances), 12 for choosing the checkpoint (4,411) and 24 for the test above (8,748). The split is by speaker only, so many dev and test sentences are also read by someone in training (see Results). English words written in Sinhala script in its transcripts were rewritten in English letters.
• —Replay, to keep the other languages: 30,128 recordings from FLEURS (English, Chinese, Spanish, French, German) and LibriSpeech train-clean-100, each labelled by the base model itself, passed twice an epoch (26% of what the model heard).
• —Recipe: a full fine-tune, audio encoder included, for one epoch of 1,816 steps of 128 utterances. AdamW at a peak rate of 2×10⁻⁵ with 2% warm-up and linear decay, gradients clipped at 1.0, bfloat16 compute with float32 master weights, on a MacBook Pro with an M4 Pro and 48 GB, with MLX Swift.
• —Checkpoint: the end of the epoch, blended with the base model: 80% of each fine-tuned weight plus 20% of the original's (weight-space ensembling). On the development sets that kept English within 0.3 points of the base model, at a cost of 0.4 points of Sinhala character error rate (6.10% against 5.70% unblended). It also stopped a repetition loop the unblended model fell into on one English recording. Everything was chosen on the development sets, which share the test set's overlap with the training sentences; the test sets were used only to report this model, never to choose it.
• —Export: the text model quantised to 8 bits (affine, group 64) as mlx-community's is, the audio encoder in bfloat16, and Sinhala added to support_languages.

Learn how it was made

The project is written up as a blog series, for anyone who wants to teach a speech model a new language, or train one on a Mac. It covers each step, mistakes included:

1. 1.Measuring the models that already exist: the speech models that claim to support Sinhala, measured on 300 real recordings, and why none was usable for dictation.
2. 2.Why Qwen3-ASR as the base model: one model for every language, a decoder that can be told the language, which size, and what starting without Sinhala costs.
3. 3.Preparing the training data: English words rewritten in English letters, test speakers kept out of training, and a replay set labelled by the model itself so it keeps its other languages.
4. 4.Training on a Mac, the same way the app runs: why a MacBook Pro instead of a rented GPU, and the three places the trainer had to match the app exactly, one of them a bug behind 40% of the errors in noisy English.
5. 5.The first training run: a loss that looked perfect while half the transcripts were wrong, choosing a learning rate by watching English get worse, three precision bugs no loss curve showed, and winning English back by blending with the original model.
6. 6.What's next: what had to be true before the model shipped, and the questions still open.

Every post is at purinda.com/tags/sinhala. The trainer, the data preparation and a step-by-step runbook (docs/training.md) are in Nerdstorm/LiveTranscribe-Sinhala.

Limitations

• —OpenSLR 52 is read sentences, recorded by volunteers. Conversation, dictation, strong accents and noise are untested.
• —English words are rare in it (a few percent of sentences), so Sinhala sentences that mix in a lot of English may not come out in English letters.
• —Spanish, French and German lost more to the fine-tune than English did: 19% to 33% more word errors than the base model on FLEURS (Languages). For those languages alone, the base model is the better choice.
• —Its transcripts write numbers as words and use little punctuation, and the model's Sinhala does the same.
• —Sinhala takes Qwen's tokenizer about 8.2 tokens per second of speech, against 3.6 for English, and the model writes one token at a time, so Sinhala is slower to transcribe than English.
• —Greedy decoding can, rarely, get stuck repeating a word or phrase. This model didn't in the tests above, but the unblended one did once, on English. mlx-audio-swift's loop backstop stops only loops of 3 or fewer distinct tokens. A Sinhala word is often 8 or more, and so is a short English phrase, so such a loop runs to maxTokens (8,192 by default, about a minute). Set maxTokens from the clip's length, for example 64 plus 30 per second of audio. Sinhala speech averages about 8 tokens a second.
• —Tested with mlx-audio-swift only; mlx-audio (Python) should compute the same features but hasn't been tried.

Licence

The weights are released under CC BY-SA 4.0, because they were trained on OpenSLR 52, which is under the same licence.

The base model, Qwen3-ASR 0.6B by the Qwen team, Alibaba Cloud, is under the Apache License 2.0, a copy of which is in LICENSE-APACHE-2.0; this model is a modified version of it.

Training data:

• —Large Sinhala ASR training data set (OpenSLR 52), Copyright 2016, 2017, 2018 Google, Inc., CC BY-SA 4.0. Oddur Kjartansson, Supheakmungkol Sarin, Knot Pipatsrisawat, Martin Jansche and Linne Ha. 2018. Crowd-Sourced Speech Corpora for Javanese, Sundanese, Sinhala, Nepali, and Bangladeshi Bengali. SLTU 2018, pages 52–55.
• —FLEURS (google/fleurs), CC BY 4.0, whose speakers read sentences from FLORES (CC BY-SA 4.0).
• —LibriSpeech ASR corpus (OpenSLR 12), train-clean-100, CC BY 4.0.