thorhojhus/whisper-large-v3-turbo-danish
whisper-large-v3-turbo-danish
░▒▓█╔════════════════════════════════════════════════════════════════════╗█▓▒░
░▒▓█│ │█▓▒░
░▒▓█│ ██╗ ██╗██╗ ██╗██╗███████╗██████╗ ███████╗██████╗ │█▓▒░
░▒▓█│ ██║ ██║██║ ██║██║██╔════╝██╔══██╗██╔════╝██╔══██╗ │█▓▒░
░▒▓█│ ██║ █╗ ██║███████║██║███████╗██████╔╝█████╗ ██████╔╝ │█▓▒░
░▒▓█│ ██║███╗██║██╔══██║██║╚════██║██╔═══╝ ██╔══╝ ██╔══██╗ │█▓▒░
░▒▓█│ ╚███╔███╔╝██║ ██║██║███████║██║ ███████╗██║ ██║ │█▓▒░
░▒▓█│ ╚══╝╚══╝ ╚═╝ ╚═╝╚═╝╚══════╝╚═╝ ╚══════╝╚═╝ ╚═╝ │█▓▒░
░▒▓█│ │█▓▒░
░▒▓█│ ░░▒▒▓▓ D A N I S H S P E E C H ▓▓▒▒░░ │█▓▒░
░▒▓█│ │█▓▒░
░▒▓█├────────────────────────────────────────────────────────────────────┤█▓▒░
░▒▓█│ R E L E A S E I N F O [ MODEL / WEIGHTS ] │█▓▒░
░▒▓█├────────────────────────────────────────────────────────────────────┤█▓▒░
░▒▓█│ │█▓▒░
░▒▓█│ EDITION .......... LARGE-V3-TURBO │█▓▒░
░▒▓█│ LANGUAGE ......... DANISH │█▓▒░
░▒▓█│ STATUS ........... SOTA-ish │█▓▒░
░▒▓█│ LAYERS ........... 32 ENCODER / 4 DECODER │█▓▒░
░▒▓█│ MODEL WIDTH ...... 1280 │█▓▒░
░▒▓█│ AUDIO INPUT ...... 16 kHz MONO / 128 MEL BINS │█▓▒░
░▒▓█│ TRAIN WINDOW ..... 30 SECONDS │█▓▒░
░▒▓█│ VOCABULARY ....... 51,866 TOKENS │█▓▒░
░▒▓█│ OPTIMIZER ........ MUON + ADAMW │█▓▒░
░▒▓█│ TRAINING ......... 200,000 OPTIMIZER STEPS │█▓▒░
░▒▓█│ TRAINED ON ....... NVIDIA GEFORCE RTX 4090 │█▓▒░
░▒▓█│ │█▓▒░
░▒▓█├────────────────────────────────────────────────────────────────────┤█▓▒░
░▒▓█│ 01 USAGE / 02 TRAINING / 03 EVALUATION │█▓▒░
░▒▓█│ 04 LIMITATIONS / 05 LICENSE / 06 CREDITS │█▓▒░
░▒▓█│ │█▓▒░
░▒▓█╚════════════════════════════════════════════════════════════════════╝█▓▒░
░░▒▒▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▒▒░░This is a Danish continuation of `openai/whisper-large-v3-turbo`. The architecture and tokenizer are unchanged. The released weights are an exponential moving average taken after 200,000 optimizer steps.
01 / USAGE
import torch
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline
model_id = "thorh/whisper-large-v3-turbo-danish"
device = "cuda:0" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if torch.cuda.is_available() else torch.float32
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForSpeechSeq2Seq.from_pretrained(
model_id,
dtype=dtype,
low_cpu_mem_usage=True,
).to(device)
transcribe = pipeline(
"automatic-speech-recognition",
model=model,
tokenizer=processor.tokenizer,
feature_extractor=processor.feature_extractor,
dtype=dtype,
device=device,
)
result = transcribe(
"audio.wav",
generate_kwargs={"language": "da", "task": "transcribe", "num_beams": 1},
)
print(result["text"])02 / TRAINING
Training used the Danish portions of CoRal v3, Common Voice 26.0, FLEURS, FTSpeech and NST together with filtered pseudo-labelled data. The pseudo-labels were generated with `openai/whisper-large-v3` and confidence-filtered before use.
Targets retained their original casing, punctuation and numeral style. Audio augmentation, SpecAugment and synthetic no-speech examples were included. The data design was inspired by the weak-supervision and no-speech principles in the original Whisper paper.
The following figures are cumulative audio sampled during training. They include repeated sampling and are not unique dataset durations.
Additional recipe details are recorded in training_recipe.json.
03 / EVALUATION
The checkpoint was evaluated against the public Danish ASR Leaderboard protocol using greedy Danish transcription. Short clips used standard decoding; clips over 30 seconds used timestamped sequential long-form decoding. The leaderboard normalizer was applied, including Danish numeral normalization.
Exact revisions and decoding settings are recorded in training_recipe.json.
04 / LIMITATIONS
- Accuracy in languages other than Danish may be reduced.
- Long-form behavior was not directly trained and should be validated separately.
05 / LICENSE
The model weights are released under the MIT License. Training data are not redistributed and remain subject to their source terms. See THIRD_PARTY_NOTICES.md.
06 / CREDITS
@article{radford2022robust,
title={Robust Speech Recognition via Large-Scale Weak Supervision},
author={Radford, Alec and Kim, Jong Wook and Xu, Tao and Brockman, Greg and McLeavey, Christine and Sutskever, Ilya},
journal={arXiv preprint arXiv:2212.04356},
year={2022}
}