Modelpublic

Nkugwa/whisper-ugandan-english-dysarthric-v8

sourceHugging Faceapache-2.0updated 6mo agoView on Hugging Face

0likes10downloads

Model Card

Whisper Ugandan English Dysarthric Speech Recognition (v8)

A fine-tuned Whisper-Small model for Ugandan English speakers with dysarthria, using LoRA (Low-Rank Adaptation) for parameter-efficient fine-tuning.

Model Description

—Base model: openai/whisper-small
—Fine-tuning method: LoRA (r=128, alpha=256)
—Fine-tuned on: cdli/ugandan_english_nonstandard_speech_v1.0
—Language: Ugandan English (en)

Performance

Model	WER
Base Whisper-Small (no fine-tuning)	~75%
This model (v8)	32.13%

Training Data

—Real dysarthric samples: 5,172
—Augmented samples: 46,000 (16 medical dysarthric augmentation types)
—Total: 51,172 samples

LoRA Configuration

python

LoraConfig(
    r=128,
    lora_alpha=256,
    target_modules=["q_proj","k_proj","v_proj","o_proj","fc1","fc2",
                    "encoder_attn.q_proj","encoder_attn.k_proj",
                    "encoder_attn.v_proj","encoder_attn.out_proj"],
    lora_dropout=0.1,
    bias="none"
)

How to Use

python

from transformers import WhisperProcessor, WhisperForConditionalGeneration
from peft import PeftModel
import torch

# Load base model + LoRA adapter
base_model = WhisperForConditionalGeneration.from_pretrained("openai/whisper-small")
model = PeftModel.from_pretrained(base_model, "nugwa-mark/whisper-ugandan-english-dysarthric-v8")
processor = WhisperProcessor.from_pretrained("nugwa-mark/whisper-ugandan-english-dysarthric-v8")

import librosa
audio, sr = librosa.load("your_audio.wav", sr=16000)
inputs = processor(audio, sampling_rate=16000, return_tensors="pt")
with torch.no_grad():
    generated = model.generate(
        inputs.input_features,
        num_beams=10,
        repetition_penalty=1.1,
    )
transcription = processor.decode(generated[0], skip_special_tokens=True)
print(transcription)

Offline Usage

python

# Save locally
processor.save_pretrained("./english-asr-offline")
model.save_pretrained("./english-asr-offline")

Intended Use

Same as Luganda model — assistive technology for Ugandan dysarthric speakers in healthcare, education, and daily communication contexts.

Citation

@misc{ugandan-english-dysarthric-asr-2026,
  title={Ugandan English Dysarthric Speech Recognition},
  author={Nugwa Mark},
  year={2026},
  publisher={HuggingFace},
  url={https://huggingface.co/nugwa-mark/whisper-ugandan-english-dysarthric-v8}
}