tonibirat/sagarmatha-v4-nepali-asr

Sagarmatha V4 - Nepali Automatic Speech Recognition

Sagarmatha V4 is a fine-tuned version of openai/whisper-large-v3-turbo for Nepali (Devanagari script) automatic speech recognition. It was trained on 265 hours of curated Nepali audio using parameter-efficient fine-tuning (QLoRA) and achieves a Word Error Rate of 30.99% on a held-out test set, representing approximately a 20 percentage point improvement over the zero-shot baseline.

Model Details

Model Description

This repository contains the fully merged model: the LoRA adapter weights have been folded into the base model parameters. No PEFT library is required at inference time.

Intended Use

• —Transcription of spoken Nepali audio to Devanagari text
• —Research on low-resource South Asian language speech recognition
• —Downstream NLP pipelines requiring Nepali ASR

Out-of-Scope Use

• —Translation tasks (the model was fine-tuned for transcription only)
• —Languages other than Nepali
• —Real-time streaming transcription without chunking for audio longer than 30 seconds

Evaluation Results

Evaluated on held-out splits of the Sagarmatha V4 dataset after 10,000 training steps (~3.5 epochs). Two splits were used: a random sample across all clip lengths, and a long-clip split restricted to utterances of more than 25 words.

The near-identical WER between short and long clips confirms that the EOS truncation bias present in earlier model versions (Sagarmatha V3, WER 55.74% with 44% deletions on long clips) has been fully resolved in this version.

Usage

Direct Inference

from transformers import pipeline

pipe = pipeline(
    "automatic-speech-recognition",
    model="tonibirat/sagarmatha-v4-nepali-asr",
    generate_kwargs={"language": "nepali", "task": "transcribe"},
    device=0,  # use -1 for CPU
)

result = pipe("audio.wav")
print(result["text"])

Long-Form Audio (recommended for audio longer than 30 seconds)

result = pipe(
    "long_audio.wav",
    return_timestamps=True,
    chunk_length_s=30,
    stride_length_s=5,
)
print(result["text"])

for chunk in result["chunks"]:
    start, end = chunk["timestamp"]
    print(f"[{start:.1f}s - {end:.1f}s]  {chunk['text']}")

With WhisperProcessor Directly

import torch
from transformers import AutoProcessor, WhisperForConditionalGeneration

processor = AutoProcessor.from_pretrained("tonibirat/sagarmatha-v4-nepali-asr")
model = WhisperForConditionalGeneration.from_pretrained(
    "tonibirat/sagarmatha-v4-nepali-asr",
    torch_dtype=torch.float16,
    device_map="auto",
)

# Prepare audio (16 kHz mono expected)
inputs = processor(audio_array, sampling_rate=16000, return_tensors="pt")
predicted_ids = model.generate(
    inputs["input_features"].to(model.device, dtype=torch.float16),
    language="nepali",
    task="transcribe",
)
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)
print(transcription[0])

Training Details

Dataset

The model was trained on Sagarmatha V4, a curated corpus of 265 hours of Nepali speech comprising 156,375 audio clips in FLAC format.

Data from the OpenSLR source was obtained from openslr.org/54 (Nepali TTS and ASR corpus).

Hyperparameters

Hardware

Limitations

• —Accent coverage: The model was trained predominantly on standard spoken Nepali. Performance on strong regional accents or dialects has not been evaluated.
• —Code-switching: Utterances mixing Nepali and English may produce degraded output, as the training corpus does not contain code-switched speech.
• —Noisy environments: No noise augmentation was applied during training. Performance in high-noise conditions is expected to be lower.
• —Long-form audio: Clips longer than 30 seconds require chunked inference. End-to-end transcription of arbitrarily long audio without chunking is not supported.
• —Translation: The model was not fine-tuned for translation. Invoking task="translate" will produce Whisper's default translation behaviour, not a fine-tuned translation system.

Citation

If you use this model in your research, please cite the following:

@misc{birat2026sagarmatha,
  author       = {Birat, Toni},
  title        = {Sagarmatha V4: QLoRA Fine-Tuning of Whisper for Low-Resource Nepali ASR},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/tonibirat/sagarmatha-v4-nepali-asr}},
}

Acknowledgements

Base model weights are from openai/whisper-large-v3-turbo. Training was conducted on Kaggle (free GPU tier, T4 hardware) using the Hugging Face PEFT and Transformers libraries.