CoolFace
Modelpublic

vanshnawander/whisper-small-telugu

sourceHugging Faceapache-2.0updated 9mo agoView on Hugging Face
0likes26downloads
Model Card

vanshnawander/whisper-small-telugu

This is a fine-tuned version of openai/whisper-small for Telugu automatic speech recognition (ASR).

Model Description

  • —Base Model: openai/whisper-small
  • —Language: Telugu (te)
  • —Task: Automatic Speech Recognition (transcribe)
  • —Training Data: ai4bharat/Kathbath
  • —Fine-tuning Framework: Transformers + Custom DALI Pipeline

Training Details

The model was fine-tuned on the Kathbath Telugu dataset with the following configuration:

  • —Epochs: 3
  • —Batch Size: 16 (effective ~96 with gradient accumulation)
  • —Learning Rate: 1e-5
  • —Mixed Precision: FP16
  • —Gradient Checkpointing: Enabled

Evaluation Results

Evaluated on the Shrutilipi benchmark - a large-scale ASR dataset for Indian languages.

ModelWERCERImprovement
Base (openai/whisper-small)N/A%N/A%-
This Model69.7%28.9%

Usage

Basic Usage

python
from transformers import WhisperProcessor, WhisperForConditionalGeneration
import librosa

# Load model and processor
processor = WhisperProcessor.from_pretrained("vanshnawander/whisper-small-telugu")
model = WhisperForConditionalGeneration.from_pretrained("vanshnawander/whisper-small-telugu")

# Load audio
audio, sr = librosa.load("audio.wav", sr=16000)

# Transcribe
input_features = processor(audio, sampling_rate=16000, return_tensors="pt").input_features
generated_ids = model.generate(input_features, language="te", task="transcribe")
transcription = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]

print(transcription)

Using Pipeline

python
from transformers import pipeline

pipe = pipeline(
    "automatic-speech-recognition",
    model="vanshnawander/whisper-small-telugu",
    chunk_length_s=30,
)

result = pipe("audio.wav", generate_kwargs={"language": "te", "task": "transcribe"})
print(result["text"])

Limitations

  • —Optimized for Telugu speech; may not perform well on other languages
  • —Best performance on clear audio with minimal background noise
  • —May struggle with very fast speech or heavy code-mixing

Citation

If you use this model, please cite:

bibtex
@misc{vanshnawander_whisper_small_telugu},
  author = {Vansh Nawander},
  title = {vanshnawander/whisper-small-telugu},
  year = {2025},
  publisher = {HuggingFace},
  url = {https://huggingface.co/vanshnawander/whisper-small-telugu}
}

Acknowledgments