CoolFace
Modelpublic

pathslash/whisper-small-gujarati

sourceHugging Faceapache-2.0updated 5mo agoView on Hugging Face
0likes
Model Card

Whisper Small – Gujarati ASR

Fine-tuned openai/whisper-small for Gujarati automatic speech recognition (speech-to-text).

Training Details

ParameterValue
Base modelopenai/whisper-small (244M params)
Dataset`shunyalabs/gujarati-speech-dataset` (~20GB)
Training methodFull fine-tuning (no LoRA)
Learning rate1e-5 (linear decay)
Warmup steps500
Max steps10,000
Effective batch size32 (16 × 2 grad accum)
PrecisionFP16
OptimizerAdamW
Gradient checkpointing✅

Recipe Sources

  • —IIT Madras Whisper Hindi recipe (vasista22/whisper-hindi-large-v2)
  • —Paper: Fine-tuning Whisper for Pashto ASR (arxiv: 2604.06507) — confirms full FT >> LoRA for low-resource
  • —Paper: Enhancing Whisper's Accuracy for Indian Languages (arxiv: 2412.19785)

Usage

python
from transformers import pipeline

transcriber = pipeline(
    "automatic-speech-recognition",
    model="pathslash/whisper-small-gujarati",
)

result = transcriber("path/to/gujarati_audio.wav")
print(result["text"])

With explicit Gujarati language forcing:

python
from transformers import WhisperForConditionalGeneration, WhisperProcessor

processor = WhisperProcessor.from_pretrained("pathslash/whisper-small-gujarati")
model = WhisperForConditionalGeneration.from_pretrained("pathslash/whisper-small-gujarati")

# Load your audio (must be 16kHz mono)
import librosa
audio, sr = librosa.load("path/to/audio.wav", sr=16000)

input_features = processor(audio, sampling_rate=16000, return_tensors="pt").input_features
predicted_ids = model.generate(input_features)
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print(transcription)

Training Script

The full training script is available in this repo: `train_whisper_gujarati.py`

To run training yourself:

bash
pip install transformers datasets evaluate jiwer torch accelerate trackio soundfile librosa
python train_whisper_gujarati.py

Recommended hardware: NVIDIA A10G (24GB) or better.