smainye/whisper-small-kenyan-english-nonstandard-v2

Whisper-small-kenyan-english-nonstandard-v2 (Non-Standard, Noise-Robust)

Fine-tuned from `openai/whisper-small` on non-standard Kenyan English speech, with waveform-level noise augmentation to improve robustness under real-world Kenyan field deployment conditions (mobile networks, ambient crowd noise, device volume variation).

Developed by Alexander Sheldon Mainye.

Model Details

Training Report


Performance

Evaluated on the held-out test split of cdli/kenyan_english_nonstandard_speech_v1.0 (926 examples). Clean evaluation uses unaugmented audio; noisy evaluation applies the same waveform augmentation pipeline used during training.

The 5.85 pp robustness gap represents a 56% relative reduction compared to earlier runs trained at higher augmentation probability (prior best: 13.5 pp gap), while clean test WER remained competitive.

Training Details

Dataset

cdli/kenyan_english_nonstandard_speech_v1.0 — nnon-standard Kenyan English speech covering etiologies.

Augmentation Pipeline

Waveform-level augmentation was applied during training only (AUGMENT_PROB = 0.4). Evaluation always used clean audio for reproducible WER/CER comparisons.

SpecAugment was also applied during training (mask_time_prob = 0.05, mask_feature_prob = 0.05).

Hyperparameters

Training ran the full 2,000-step budget without early stopping triggering. The best validation WER (21.65%) was achieved at step 1,800, confirming that patience 7 and the polynomial decay schedule were appropriate for this dataset size.

Usage

from transformers import WhisperProcessor, WhisperForConditionalGeneration
import torch

processor = WhisperProcessor.from_pretrained("smainye/whisper-small-kenyan-english-nonstandard")
model = WhisperForConditionalGeneration.from_pretrained("smainye/whisper-small-kenyan-english-nonstandard")

# Load your audio (must be 16kHz, mono)
# audio is a numpy array or torch tensor of shape (samples,)
inputs = processor(audio, sampling_rate=16000, return_tensors="pt")

with torch.no_grad():
    predicted_ids = model.generate(
        inputs["input_features"],
        language="en",
        task="transcribe"
    )

transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)
print(transcription[0])

Notes on intended use

• —Optimised for non-standard Kenyan English.
• —Most robust to ambient crowd noise and GSM/mobile codec artifacts (as trained).
• —Not fine-tuned for standard American or British English; general-purpose WER on those varieties may be lower than openai/whisper-small baseline.
• —Evaluation was conducted on a noise-augmented test set simulating field conditions; real-world WER will vary based on recording quality and acoustic environment.

Limitations

• —Dataset size: the training set comprises 4,243 examples. Performance on speaker demographics, dialects, or acoustic conditions not well-represented in cdli/kenyan_english_nonstandard_speech_v1.0 may degrade.
• —Room reverb not covered: reverb-heavy environments (lecture halls, large rooms) were not part of the augmentation pipeline in this run. Performance in strongly reverberant conditions is untested.
• —Model capacity: based on whisper-small (241M parameters). Larger Whisper variants may yield lower WER on this domain.
• —Language: Suited for Kenyan English speakers.

Training Infrastructure

Training was conducted using Modal GPU infrastructure with the Hugging Face transformers Seq2SeqTrainer.

Citation

If you use this model in your work, please cite it as:

@misc{whisper-small-kenyan-english-nonstandard,
  author       = {Mainye, Alexander Sheldon},
  title        = {Whisper Small Fine-tuned on Non-Standard Kenyan English Speech},
  year         = {2026},
  publisher    = {Hugging Face},
  url          = {https://huggingface.co/smainye/whisper-small-kenyan-english-nonstandard}
}