Dr-AliGomaa/whisper-large-v3-ar-eg-timestamps

whisper-large-v3-ar-eg-timestamps

Same model as `whisper-large-v3-ar-eg` — same accuracy — but it keeps Whisper's timestamp prediction, so every transcript stays anchored to the recording.

That is not automatic. Naively fine-tuning Whisper on untimestamped text erases timestamp prediction entirely — catastrophic forgetting of the timestamp task. It survives here because timestamp-supervised data from `ar-quran-cm17-timestamps` was mixed into 23.4 % of the training rows.

Why it matters for this domain. If people read a machine transcript instead of listening to the recitation, timestamps are what let them go back and check: select a verse, hear exactly that span, verify it. A specialist that transcribes sacred text accurately and keeps it anchored to the audio is better suited to faithful preservation than one that emits text alone. The strongest open Arabic generalist (Cohere Transcribe Arabic) does not expose timestamps.

• —Paper: A Quran and Hadith Speech Resource and Benchmark for Arabic ASR (preprint, DOI) · Code: github.com/DrAliGomaa/quran-hadith-asr
• —Created by Mohamed Kotb · Under the patronage of Prof. Ali Gomaa, former Grand Mufti of Egypt, who reviewed the handling of the religious material.

Results

Identical to `whisper-large-v3-ar-eg` — timestamp supervision costs no accuracy. Normalized WER/CER (%), scored with `eval/asr_score.py`.

Text accuracy is measured with return_timestamps=False; the timestamps are an additional output, not a different transcript.

Usage — use word timestamps

This is the one place where usage differs from the other two models.

Recommended: word-level

import torch
from transformers import pipeline

pipe = pipeline(
    "automatic-speech-recognition",
    model="Dr-AliGomaa/whisper-large-v3-ar-eg-timestamps",
    torch_dtype=torch.float16,
    device="cuda:0",
    chunk_length_s=30,
)

gen = {
    "language": "arabic",
    "task": "transcribe",
    "num_beams": 5,
    "temperature": (0.2),
    "condition_on_prev_tokens": False,
    "compression_ratio_threshold": 1.35,
    "logprob_threshold": -1.0,
    "max_new_tokens": 444,
}

out = pipe("lecture.mp3", return_timestamps="word", generate_kwargs=gen)

for ch in out["chunks"][:10]:
    start, end = ch["timestamp"]
    print(f"{start:7.2f} → {end:7.2f}   {ch['text']}")

Segment-level works, but the boundaries are coarse

out = pipe("lecture.mp3", return_timestamps=True, generate_kwargs=gen)

Segments can run 15–20 seconds long. Whisper emits a segment boundary where it decides a phrase ended, and on recitation — which is continuous, melodic and sparsely punctuated — those decisions land far apart. If you need sentence- or verse-level alignment, do not ask for segment timestamps. Ask for word timestamps and group them yourself; you control the rule and the result is far tighter.

Grouping words into sentences

def group_words(chunks, max_gap=0.6, max_dur=12.0):
    """Word chunks -> sentence-ish spans. Break on a silence gap or on length."""
    spans, cur = [], None
    for ch in chunks:
        start, end = ch["timestamp"]
        if start is None or end is None:      # dropped word, keep the text
            if cur: cur["text"] += ch["text"]
            continue
        if cur is None:
            cur = {"start": start, "end": end, "text": ch["text"]}
            continue
        gap = start - cur["end"]
        if gap > max_gap or (end - cur["start"]) > max_dur:
            spans.append(cur)
            cur = {"start": start, "end": end, "text": ch["text"]}
        else:
            cur["end"] = end
            cur["text"] += ch["text"]
    if cur: spans.append(cur)
    return spans

for s in group_words(out["chunks"]):
    print(f"[{s['start']:.2f}–{s['end']:.2f}] {s['text'].strip()}")

Tune max_gap to the material: recitation pauses between verses are long, so 0.6–1.0 s works well; conversational speech needs a smaller value.

Notes

• —Word timestamps are quantized to Whisper's native 20 ms resolution.
• —A word may come back with timestamp = (None, None) if the model dropped the alignment — keep the text, as the snippet above does, rather than dropping the word.
• —For plain text with no timestamps at all, pass return_timestamps=False and you get exactly the behaviour of whisper-large-v3-ar-eg.

Two things that will otherwise cost you accuracy

1. Run the audio pipeline first — silence-trim, loudness-normalize to −16 LUFS, edge-pad 100 ms, gain-correct, segment ≤ 30 s. This matters doubly here: timestamps are relative to the audio you feed in, so trimming silence after transcription shifts every one of them. `pipeline/`.

2. Score with `eval/asr_score.py` — Arabic WER moves materially with the normalizer.

Training

Full recipe: `training/training.py`.

Intended use and limits

• —For transcription assistance and research. Not an authority on the correct text of the Quran or hadith — verify against canonical written sources before any religious use.
• —Output is *Imlāʾī orthography without tashkīl*; numerals are Arabic words**.
• —Not a general Egyptian-dialect model — one speaker, formal register.
• —Timestamps are a navigation aid, not forced alignment: they are good enough to jump to a verse and listen, not to cut audio to the millisecond.
• —Religious content reviewed and approved by Prof. Ali Gomaa, former Grand Mufti of Egypt and member of Al-Azhar's Council of Senior Scholars, under whose patronage this work was carried out.

Citation

@misc{kotb2026quranhadith,
  title  = {A Quran and Hadith Speech Resource and Benchmark for Arabic ASR,
            with Professional-Reciter Training and Validation},
  author = {Mohamed Kotb},
  year   = {2026},
  publisher = {Zenodo},
  doi    = {10.5281/zenodo.21927416},
  url    = {https://doi.org/10.5281/zenodo.21927416},
  note   = {Preprint}
}