sapinsapin/whisper-large-v3-pld-fil-norm

whisper-large-v3-pld-fil-norm

Filipino speech recognition: `openai/whisper-large-v3` finetuned on the Philippine Language Dataset (PLD), read speech collected by the UP Diliman Digital Signal Processing Laboratory. Part of the halohalo project.

In plain words

This model turns recorded Filipino speech into text. Give it an audio clip (a WAV file, 16 kHz, one channel), and it returns what was said.

How good is it? Tested on sentences it had never seen, spoken by people it had never heard, it gets about 95 of every 100 characters and 88 of every 100 words right. Word accuracy is the one you will notice.

It writes everything in lowercase, without punctuation or accent marks. If you need those, they have to be added afterwards.

It was trained on clear, read speech recorded for a corpus. Expect worse results on conversations, phone calls, music in the background, or a speaker switching languages mid-sentence.

This is the one to use for Filipino speech recognition from this organisation, as of the date on this card.

Method (for practitioners)

Normalised text. Trained and scored on transcripts with stress accents and punctuation removed (halolib.finetune.normalisetext). PLD marks stress on about a third of words and an ASR model is not asked for it; scoring the *same* hypotheses with and without them moved whisper-large-v3 on Cebuano from 36.9 to 24.2 WER. Continued for 5000 steps from None, whose encoder had already seen this audio — a label change does not need a restart. Outputs are lowercase with no punctuation or accents. Trained for 5000 steps on 25000 clips, frozen-disjoint split (speakers and prompts unseen in training). The Whisper arm of the R1 bake-off in docs/pldsota_track.md. Trained on PLD, which is CC-BY-NC and research-only.

Evaluation

Frozen speaker- and prompt-disjoint split of PLD (splits/pld_*.json in the repo): no test speaker and no test sentence appears anywhere in training. Numbers on this split are not comparable with the in-domain figures on the dataset card, which share both, and are typically several times higher for that reason.

cer and wer are character and word error rates on the whole test split, hypothesis and reference both lowercased, whitespace-normalised, and with stress accents and punctuation removed (halolib.finetune.normalise_text). PLD marks stress on about a third of words and an ASR model is not asked for it: scoring the same hypotheses with and without them moved whisper-large-v3 on Cebuano from 36.9 to 24.2 WER. eval_loss is the training objective on the test split.

How to use

from transformers import WhisperForConditionalGeneration, WhisperProcessor
import torch, soundfile as sf

proc = WhisperProcessor.from_pretrained("sapinsapin/whisper-large-v3-pld-fil-norm")
model = WhisperForConditionalGeneration.from_pretrained("sapinsapin/whisper-large-v3-pld-fil-norm").eval()
wav, sr = sf.read("clip.wav")          # 16 kHz mono
feats = proc(wav, sampling_rate=16000, return_tensors="pt").input_features
with torch.no_grad():
    ids = model.generate(feats, task="transcribe", max_new_tokens=200)
print(proc.batch_decode(ids, skip_special_tokens=True)[0])

Languages other than Filipino and English were trained under Whisper's <|tl|> token, the closest one it has; do not pass a language= argument. Output is lowercase with no punctuation or accents.

Caveats

• —Trained on read, prompted speech; accuracy drops on spontaneous or noisy audio.
• —One corpus, one recording setup. Cross-corpus tests on Filipino showed large drops for models of this kind.

Licence

cc-by-nc-4.0. PLD is CC-BY-NC and research-only, so every model trained on it inherits that regardless of the base model's own licence. The base model's terms apply in addition.

Trained with finetune_asr.py from halohalo; the dataset adapter normalises each corpus to (audio@16k, text, speaker_id).