dev-ahmedhany/whisper-large-v3-turbo-arabic-ft-ct2-int8

Whisper-large-v3-turbo — Arabic 4-dialect FT (CT2 int8)

CTranslate2 int8 build of openai/whisper-large-v3-turbo fine-tuned with QLoRA on a dialect-balanced Arabic mix (MSA + Egyptian + Levantine + Gulf). 820 MB on disk, runs at real-time on commodity CPU.

This is the production-deployable artifact (model.bin + tokenizer + normalizer). For the float16/bf16 PyTorch version, see `whisper-large-v3-turbo-arabic-ft`; for the LoRA adapter (further fine-tuning), see `whisper-large-v3-turbo-arabic-ft-lora`. Note (2026-05): the -lora and -ft repos linked above currently host v1 weights (LoRA r=32). The int8 here is v2 (r=8, the recipe that survives int8 quantization without WER regression). See paper §6.2/§6.5.

Headline WER on mixed-domain test sets (n=100/dialect, int8, beam=2, threads=8, c3-standard-8)

50% Casablanca + 50% broadcast (MGB-3 / MASC) for Egyptian and Levantine; 100% Casablanca UAE for Gulf (no public broadcast Gulf source); 100% FLEURS broadcast for MSA. Same exact recordings + decoding config used for both rows.

v2-ft wins by 3.35 pp average on the mixed-domain test, with dominant gains on Egyptian (−8.52) and Gulf (−5.08). The signed-test-set version (paper §6.5, Casablanca-only) showed only −0.72 pp average; the mixed-domain test reveals the model's real advantage on dialect-diverse traffic that broadcast-only or Casablanca-only tests obscure.

Quickstart

pip install faster-whisper
huggingface-cli download dev-ahmedhany/whisper-large-v3-turbo-arabic-ft-ct2-int8 \
    --local-dir ./whisper-ar-int8

from faster_whisper import WhisperModel

model = WhisperModel(
    "./whisper-ar-int8",
    device="cpu", compute_type="int8", cpu_threads=8,  # 8 threads recommended
)
segments, info = model.transcribe(
    "audio.wav",
    beam_size=2,        # paper §6.6 sweet spot — best avg WER per RTF
    language="ar",
    task="transcribe",
)
print(" ".join(s.text for s in segments))

Inference RTF ≈ 0.5–0.6 (MSA) to 1.4–1.7 (dialects) on a c3-standard-8 (Intel Sapphire Rapids), peak RAM ~1.4 GB. For lower-latency real-time captioning, set beam_size=1 (≈30% faster, ~1pp WER cost).

Training recipe (recap)

• —Base: openai/whisper-large-v3-turbo
• —QLoRA: NF4 + bf16 compute, r=8, α=16, dropout 0.05, target modules = q/v/k/out_proj + fc1/fc2
• —Optimizer: pagedadamw8bit, lr = 1e-4, warmup 0.1
• —Effective batch 16 (per-device 8 × grad-accum 2), gradient_checkpointing
• —Max 10,000 steps, early-stop patience=4 evals (best at step ~2000, val WER 28.60%)
• —load_best_model_at_end=True so the saved adapter is the best-WER checkpoint
• —Training data: 3,900 rows (~7 h) — 51% MSA (Common Voice 18) + Casablanca validation splits for Egyptian/Jordanian/UAE (used as train data — Casablanca has no train split)
• —Maghrebi excluded (84.7% zero-shot WER too far gone for this training budget — paper §3.7)

Conversion to CT2 int8

ct2-transformers-converter \
    --model checkpoints/v2-merged \
    --output_dir checkpoints/v2-ct2-int8 \
    --quantization int8 \
    --copy_files preprocessor_config.json tokenizer_config.json normalizer.json \
                 special_tokens_map.json added_tokens.json merges.txt vocab.json tokenizer.json

Limitations

• —Maghrebi is out of scope. This model was not trained on Algerian/Moroccan/Tunisian dialects; expect zero-shot-quality (≥80% WER) on those.
• —MSA: zero-shot turbo is slightly better. If your traffic is overwhelmingly broadcast Arabic and you don't need dialect support, plain zero-shot turbo is the right pick (10.20% MSA vs 11.42%).
• —Gulf test is Casablanca-only: no public broadcast Gulf corpus exists at sufficient quality for our 50/50 mix. Treat the Gulf number as conversational-only.
• —Single-utterance assumed. Audio longer than 30s is internally chunked by faster-whisper; multi-speaker diarization is not provided.

Citation

@misc{hany2026whisperarabic,
  title        = {Production-Aware Fine-Tuning of Whisper Variants for Multi-Dialect
                  Arabic ASR: A Cross-Platform CPU Inference Study},
  author       = {Hany, Ahmed},
  year         = {2026},
  howpublished = {Preprint, arXiv (in preparation)},
  url          = {https://github.com/dev-ahmedhany/whisper-arabic-dialects},
}

License

Apache-2.0 (inherits from base openai/whisper-large-v3-turbo).