sapinsapin/whisper-large-v3-pld-pam

whisper-large-v3-pld-pam

Kapampangan speech recognition: `openai/whisper-large-v3` finetuned on the Philippine Language Dataset (PLD), read speech collected by the UP Diliman Digital Signal Processing Laboratory. Part of the halohalo project.

In plain words

This model turns recorded Kapampangan speech into text. Give it an audio clip (a WAV file, 16 kHz, one channel), and it returns what was said.

How good is it? Tested on sentences it had never seen, spoken by people it had never heard, it gets about 94 of every 100 characters and 77 of every 100 words right. Word accuracy is the one you will notice.

It was trained on clear, read speech recorded for a corpus. Expect worse results on conversations, phone calls, music in the background, or a speaker switching languages mid-sentence.

Not sure which model to pick? For Kapampangan, use `whisper-large-v3-pld-pam-norm` — it is the most accurate one this organisation has published. This one exists for comparison and research.

Method (for practitioners)

Trained for 5000 steps on 25000 clips, frozen-disjoint split (speakers and prompts unseen in training). The Whisper arm of the R1 bake-off in docs/pldsotatrack.md. Trained on PLD, which is CC-BY-NC and research-only.

Evaluation

Frozen speaker- and prompt-disjoint split of PLD (splits/pld_*.json in the repo): no test speaker and no test sentence appears anywhere in training. Numbers on this split are not comparable with the in-domain figures on the dataset card, which share both, and are typically several times higher for that reason.

cer and wer are character and word error rates on the whole test split, hypothesis and reference both lowercased and whitespace-normalised. Accents and punctuation in the reference count as errors when missing, which is strict: PLD marks stress on about a third of words. The -norm variant of this model uses the other convention. eval_loss is the training objective on the test split.

How to use

from transformers import WhisperForConditionalGeneration, WhisperProcessor
import torch, soundfile as sf

proc = WhisperProcessor.from_pretrained("sapinsapin/whisper-large-v3-pld-pam")
model = WhisperForConditionalGeneration.from_pretrained("sapinsapin/whisper-large-v3-pld-pam").eval()
wav, sr = sf.read("clip.wav")          # 16 kHz mono
feats = proc(wav, sampling_rate=16000, return_tensors="pt").input_features
with torch.no_grad():
    ids = model.generate(feats, task="transcribe", max_new_tokens=200)
print(proc.batch_decode(ids, skip_special_tokens=True)[0])

Languages other than Filipino and English were trained under Whisper's <|tl|> token, the closest one it has; do not pass a language= argument.

Caveats

• —Trained on read, prompted speech; accuracy drops on spontaneous or noisy audio.
• —One corpus, one recording setup. Cross-corpus tests on Filipino showed large drops for models of this kind.

Licence

cc-by-nc-4.0. PLD is CC-BY-NC and research-only, so every model trained on it inherits that regardless of the base model's own licence. The base model's terms apply in addition.

Trained with finetune_asr.py from halohalo; the dataset adapter normalises each corpus to (audio@16k, text, speaker_id).