Modelpublic

dev-ahmedhany/whisper-large-v3-arabic-ft-v3-lora

sourceHugging Faceapache-2.0updated 5mo agoView on Hugging Face

1likes6downloads

Model Card

Whisper-large-v3 — Arabic 4-dialect FT v3 (LoRA adapter)

QLoRA adapter for openai/whisper-large-v3 fine-tuned on 38 hours of dialect-balanced Arabic. The 111 MB adapter file rather than the merged base. Use with peft to load on top of the original Whisper-large-v3, or pull the merged version at `whisper-large-v3-arabic-ft-v3` / CT2 int8 at `whisper-large-v3-arabic-ft-v3-ct2-int8` for direct inference.

Every checkpoint preserved as a Git revision

This repo is special: every single training save (40+ checkpoints, every 250 steps) is preserved as a Git commit on the main branch. Use revision=<sha> in snapshot_download or from_pretrained to fetch any specific training step.

step	revision SHA	avg-4 WER on test	notes
500	`7d6da5dbad9a`	33.52%	first eval
1000	`9330588e0ab5`	31.90%
1500	`6587814ff6e1`	30.67%	first time below 31%
2000	`8cedaf0045aa`	29.55%	first time below 30%
2250	`c9b4f475b80d`	28.56%
2500	`193b269123c9`	28.03%
2750	`aa8c5d85af10`	28.05%	best MSA (9.78%)
3500	`e4d40d0a813f`	27.75%
4000	`6ace7d869a40`	27.68%
4250	`a66bd3990a18`	28.11%	best Gulf (39.89%)
4750 ⭐	`7923fe7bc9b7`	26.63%	best avg, recommended for production
5500	`d06b54080518`	27.03%	best Levantine (29.58%)
5750	`d849f7bbf51e`	27.20%

The "main" branch points at the latest checkpoint pushed during training (around step 8250). For the best model, use revision 7923fe7bc9b7 (ckpt-4750).

Headline WER (n=100/dialect, eval at int8/beam=2/threads=8)

50% Casablanca + 50% broadcast for Egyptian/Levantine; 100% Casablanca UAE for Gulf; 100% FLEURS for MSA. Same exact recordings + decoding config used for both rows.

Dialect	Test composition	Zero-shot Whisper-large-v3	v3-ft @ ckpt-4750	Δ
MSA	FLEURS broadcast	8.51%	10.52%	+2.01 pp
Egyptian	50 Casablanca + 50 MGB-3	38.48%	23.90%	−14.58 pp ✅
Levantine	50 Casablanca JO + 50 MASC	37.70%	30.63%	−7.07 pp ✅
Gulf	Casablanca UAE	52.72%	41.46%	−11.26 pp ✅
avg-4		34.35%	26.63%	−7.72 pp ✅

Quickstart with peft

python

from huggingface_hub import snapshot_download
from peft import PeftModel
from transformers import WhisperForConditionalGeneration, WhisperProcessor
import torch

# Best checkpoint (recommended)
adapter_dir = snapshot_download(
    "dev-ahmedhany/whisper-large-v3-arabic-ft-v3-lora",
    revision="7923fe7bc9b7",  # ckpt-4750 — best avg WER
)

base = WhisperForConditionalGeneration.from_pretrained(
    "openai/whisper-large-v3", torch_dtype=torch.bfloat16
)
proc = WhisperProcessor.from_pretrained("openai/whisper-large-v3", language="arabic", task="transcribe")
model = PeftModel.from_pretrained(base, adapter_dir).to("cuda").eval()

# inference: load audio, pass through processor + model.generate(...)

For continued fine-tuning, load the same adapter and call trainer.train() against your new dataset.

Training recipe

—Base: openai/whisper-large-v3 (1.55 B params)
—QLoRA: NF4 + bf16 compute, r=8, α=16, dropout 0.05, target modules q/v/k/out_proj + fc1/fc2
—Optimizer: pagedadamw8bit, lr=1e-4, warmup ratio 0.1
—Effective batch 16 (per-device 4 × grad-accum 4), gradient_checkpointing
—maxsteps=10000, savesteps=250, hubstrategy=everysave (every save pushed here as a Git commit)
—Training data: ~38 h, 26,817 train / 924 val rows from Casablanca (5 countries) + MGB-3 + cleaned MGB-3 + MASC + Common Voice 18
—Maghrebi excluded (84.7% zero-shot WER too far gone for QLoRA budget)

Citation

bibtex

@misc{hany2026whisperarabic,
  title        = {Production-Aware Fine-Tuning of Whisper Variants for Multi-Dialect
                  Arabic ASR: A Cross-Platform CPU Inference Study},
  author       = {Hany, Ahmed},
  year         = {2026},
  howpublished = {Preprint, arXiv (in preparation)},
  url          = {https://github.com/dev-ahmedhany/whisper-arabic-dialects},
}

License

Apache-2.0 (inherits from openai/whisper-large-v3).