CoolFace
Apppublic

yagmurtuncer/turkish-asr-leaderboard

sourceHugging Facemitupdated 2d agoView on Hugging Face
4likes
App README

πŸ‡ΉπŸ‡· Turkish ASR Leaderboard 🎀

14 open Turkish speech-recognition models on one balanced test set, scored with WER, CER and KDHO, a root- and affix-aware error rate for Turkish. All values are error rates: lower is better.

🎧 Try the models on your own audio: Turkish ASR Leaderboard Demo (Whisper, Voxtral and Voxtral + Turkish LoRA).

πŸ“Š Test set (TurkASR-Bench)

πŸ“₯ Download: datasets/yagmurtuncer/turkasr-bench (gated).

743 utterances from each of 7 sources: Common Voice 17, FLEURS, Khan Academy, MediaSpeech, VoxForge, YouTube and Real-TurnTurk (spontaneous dialogue).

  • β€”Fixed seed 42, equal weight per source.
  • β€”5,200 utterances scored.

πŸ€– Models

  • β€”Whisper: tiny, base, small, medium, large-v2, large-v3, large-v3-turbo
  • β€”Turkish Whisper fine-tunes: turkmedstt/whisper-large-v3-turkish-general, selimc/whisper-large-v3-turbo-turkish
  • β€”CTC models: XLS-R 300M (Common Voice 6 and 8 fine-tunes), MMS-1B-all (Turkish adapter)
  • β€”Multilingual models: Qwen3-ASR-1.7B, SeamlessM4T-v2-Large

πŸ“ˆ Metrics

  • β€”WER / CER: corpus-level, via jiwer.
  • β€”KDHO (KΓΆk ve Ek DuyarlΔ± Hata OranΔ±): a word-level edit distance.
  • β€”A substitution costs 0 if the words are the same.
  • β€”It costs λ·d_M if the words share a root and only the affix tags differ (Ξ» = 0.5).
  • β€”It costs 1 if the roots differ, the negation differs (geldi β†’ gelmedi) or a word cannot be analyzed.
  • β€”KDHO is never higher than WER.
  • β€”Normalization: NFC β†’ Turkish lowercasing (Iβ†’Δ±, Δ°β†’i) β†’ punctuation removed (word-internal apostrophes kept). The main setting also spells out digits in Turkish (48 β†’ kΔ±rk sekiz).
  • β€”Significance: β‰ˆ marks a difference to the next model that is not significant (paired bootstrap, B = 2,000).

🧭 Tabs

TabContent
LeaderboardRanking by WER, CER or KDHO; normalization switch; significance marks
By datasetEvery model on every source
VisualizationsSize vs. WER, per-source bars, WER vs. KDHO
Error analysisShare of error types (affix, root, negation, spacing, spelling variants, …)
Speech translationEnglish speech β†’ Turkish text: direct model vs. cascade (BLEU, chrF++, COMET)
Metrics & CalculatorWER / CER / KDHO on your own texts, with a word-by-word KDHO breakdown

πŸ“« Contact

  • β€”Maintainer: yagmurtuncer
  • β€”Last updated: September 2026
  • β€”Questions or model submissions: open a discussion in this Space.