HiTZ/whisper-large-es

Whisper Large Spanish

Model summary

Whisper Large Spanish is a high-accuracy automatic speech recognition (ASR) model for Spanish (es), fine-tuned from [openai/whisper-large] on the Spanish subset of Mozilla Common Voice 13.0. It achieves a Word Error Rate (WER) of 5.1265% on the evaluation set.

This model is designed for applications that require near state-of-the-art transcription accuracy in Spanish, such as transcription of lectures, podcasts, and other high-quality recordings.

Model description

• —Architecture: Transformer-based encoder–decoder (Whisper Large)
• —Base model: openai/whisper-large
• —Language: Spanish (es)
• —Task: Automatic Speech Recognition (ASR)
• —Output: Text transcription in Spanish
• —Decoding: Autoregressive sequence-to-sequence decoding

Large model offers very high accuracy at the cost of higher computational requirements compared to Medium or Small variants.

Intended use

Primary use cases

• —High-accuracy Spanish speech transcription
• —Applications requiring transcription of long-form audio
• —Research in Spanish ASR performance and benchmarking

Limitations

• —May underperform in extremely noisy audio or with strong regional accents not well represented in the Common Voice dataset
• —High computational cost for real-time inference
• —Not suitable for legal, medical, or safety-critical applications without human review

Training and evaluation data

• —Dataset: Mozilla Common Voice 13.0 (Spanish subset)
• —Data type: Crowd-sourced read speech
• —Preprocessing:
• —Audio resampled to 16 kHz
• —Text tokenized using Whisper tokenizer
• —Removal of invalid or corrupted samples

• —Evaluation metric: Word Error Rate (WER) on held-out evaluation set

Evaluation results

Training procedure

Training hyperparameters

• —Learning rate: 1e-5
• —Optimizer: Adam (β1=0.9, β2=0.999, ε=1e-8)
• —LR scheduler: Linear
• —Warmup steps: 500
• —Training steps: 20000
• —Train batch size: 32 (gradient accumulation 2 → effective batch size 64)
• —Eval batch size: 16
• —Seed: 42

Training results (summary)

Framework versions

• —Transformers 4.33.0.dev0
• —PyTorch 2.0.1+cu117
• —Datasets 2.14.4
• —Tokenizers 0.13.3

Example usage

from transformers import pipeline

hf_model = "HiTZ/whisper-large-es"  # replace with actual repo ID
device = 0  # -1 for CPU

pipe = pipeline(
    task="automatic-speech-recognition",
    model=hf_model,
    device=device
)

result = pipe("audio.wav")
print(result["text"])

Ethical considerations and risks

• —This model transcribes speech and may process personal data.
• —Users should ensure compliance with applicable data protection laws (e.g., GDPR).
• —The model should not be used for surveillance or non-consensual audio processing.

Citation

If you use this model in your research, please cite:

@misc{dezuazo2025whisperlmimprovingasrmodels,
  title={Whisper-LM: Improving ASR Models with Language Models for Low-Resource Languages},
  author={Xabier de Zuazo and Eva Navas and Ibon Saratxaga and Inma Hernáez Rioja},
  year={2025},
  eprint={2503.23542},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}

Please, check the related paper preprint in arXiv:2503.23542 for more details.

License

This model is available under the Apache-2.0 License. You are free to use, modify, and distribute this model as long as you credit the original creators.

Contact and attribution

• —Fine-tuning and evaluation: HiTZ/Aholab - Basque Center for Language Technology
• —Base model: OpenAI Whisper
• —Dataset: Mozilla Common Voice

For questions or issues, please open an issue in the model repository.