Modelpublic

pnawani/welsh-asr-xlsr-300m

sourceHugging Faceapache-2.0updated 1mo agoView on Hugging Face

0likes36downloads

Model Card

Welsh ASR — XLS-R 300m fine-tuned on FLEURS Welsh

facebook/wav2vec2-xls-r-300m fine-tuned for Welsh speech recognition with a character-level CTC head.

Results

FLEURS Welsh test set, 1021 utterances. Zero-shot Whisper-small is scored on the same audio through an identical text-normalization path.

Model	WER ↓	CER ↓	WER, no digits	CER, no digits
Whisper-small (zero-shot)	0.5987	0.2289	0.5853	0.2149
This model	0.3991	0.1141	0.3879	0.1098
relative change	−33.3%	−50.1%	−33.7%	−48.9%

About a fifth of FLEURS utterances contain numerals, which a character-level CTC model cannot produce from audio, so results are given both overall and on the digit-free subset. Both models are always scored on the same subsets.

Usage

python

from transformers import pipeline

pipe = pipeline("automatic-speech-recognition",
                model="pnawani/welsh-asr-xlsr-300m")
print(pipe("audio.wav")["text"])

The model expects 16 kHz mono audio. Output is lowercase, without punctuation, and keeps the Welsh circumflex vowels âêîôŵŷ.

Training


Data	FLEURS Welsh, 11.3 h after filtering (2784 clips)
Steps	2100 (~12 epochs)
Batch	2 × 8 gradient accumulation (effective 16)
Learning rate	3e-4, 200 warmup steps, fp16
Hardware	one free Colab T4

Two filters matter. Clips over 30 s are dropped, costing 1.7% of the data; the 20 s cutoff common in XLS-R tutorials would discard 31%, as FLEURS utterances are unusually long. Separately, ~15% of the train split pairs a truncated clip with its full transcript — in the worst case 0.96 s of audio against 303 characters. wav2vec2 downsamples by 320, so CTC cannot emit those labels and returns infinite loss, which becomes NaN gradients within two steps. Those clips are short, so removing them costs under 1 of 12.2 hours.

Text is lowercased, apostrophes deleted (mae'r → maer) and punctuation stripped. Welsh circumflex vowels are kept; other diacritics are folded to their base letters. The vocabulary is 45 characters.

Limitations

Numerals. The model cannot transcribe spoken numbers as digits — there is no acoustic evidence mapping dwy fil ar bymtheg to 2019. Roughly 19% of the test set contains numerals.

Circumflex vowels. Only 57% of the circumflex vowels present in references are produced (259 of 457); â → a accounts for 147 substitutions. The circumflex marks vowel length in Welsh (tan "fire" vs tân "until"), which is acoustically subtle and often recoverable only from context. A CTC decoder with no language model has no mechanism for this.

Vowel confusion is the dominant error. Fine-tuning halved character substitutions (14,032 → 6,681), but vowel-for-vowel confusion still accounts for 41% of them — essentially unchanged from the baseline's 41.3%. Welsh y, u and i are acoustically close and map onto no single English vowel.

No language model. Decoding is plain CTC argmax. A KenLM decoder would likely recover a substantial part of both the circumflex and vowel-confusion errors.

Domain. FLEURS is read speech from Wikipedia-style text. Expect degradation on spontaneous or conversational Welsh, and on dialects under-represented in FLEURS.

Code

Training, evaluation and error-analysis code: https://github.com/paarthN/welsh-asr

Acknowledgments

FLEURS (Google), XLS-R (Meta AI), Hugging Face Transformers.