ai4bharat/bhili-asr-conformer-600m

Bhili ASR (Conformer 600M)

Automatic Speech Recognition (ASR) for Bhili (भीली), specifically the Dehvali Bhili dialect, an Indo-Aryan language spoken by the Bhil community in western India. This is a fine-tuned version of ai4bharat/indic-conformer-600m-multilingual, trained on ~200 hours of Bhili read, conversational, and spontaneous speech data. The ONNX version of this model can be found here.

Other Model Variants: Canary · Nemotron

Quick Start

1. Download Files

huggingface-cli download ai4bharat/bhili-asr --local-dir bhili-asr
cd bhili-asr

2. Extract Tokenizers and NeMo

tar -xzvf tokenizers.tar.gz
tar -xzvf NeMo.tar.gz

3. Install NeMo Toolkit

⚠️ Important: This model requires a custom NeMo toolkit with multilingual tokenizer support. Do NOT install NeMo from pip or the official NVIDIA repository. You must use the provided `NeMo.tar.gz` file.

cd NeMo
pip install -e .[asr]
cd ..

Verify correct NeMo is installed:

python -c "import nemo; print(nemo.__file__)"

⚠️ Output should point to your local NeMo folder, not system packages.

4. Update Model Paths

Update the tokenizer paths to match your local directory:

python update_paths.py --root_dir /full/path/to/bhili-asr/tokenizers/tokenizers_v3

This creates bhili_asr_finetune_v1_updated.nemo with correct paths.

5. Verify Setup

Your directory should look like:

bhili-asr/
├── bhili_asr_finetune_v1.nemo
├── bhili_asr_finetune_v1_updated.nemo  (created after step 4)
├── tokenizers/
│   └── tokenizers_v3/
│       ├── as_256/
│       ├── bn_256/
│       ├── hi_256/
│       ├── mr_256/
│       └── ...
├── NeMo/
└── update_paths.py

Inference

See the GitHub repository for complete setup instructions.

Requirements

• —Python 3.10
• —NeMo toolkit (from provided NeMo.tar.gz)
• —PyTorch 2.0+
• —CUDA 11.8+ (for GPU inference)
• —soundfile (pip install soundfile)

Basic Usage

import soundfile as sf
import torch
from nemo.collections.asr.models import EncDecHybridRNNTCTCBPEModel

model = EncDecHybridRNNTCTCBPEModel.restore_from("bhili_asr_finetune_v1_updated.nemo", map_location="cuda")
model.eval()

# Load audio (must be 16kHz mono)
audio, sr = sf.read("audio.wav")
assert sr == 16000, "Audio must be 16kHz"

signal = torch.tensor(audio, dtype=torch.float32).unsqueeze(0).cuda()
signal_len = torch.tensor([signal.shape[1]]).cuda()

with torch.no_grad():
    encoded, encoded_len = model.forward(input_signal=signal, input_signal_length=signal_len)
    hyps, _ = model.decoding.rnnt_decoder_predictions_tensor(
        encoded, encoded_len, return_hypotheses=False, lang_ids=["mr"],
    )

print(hyps[0])

Because this is a multilingual model with per-language joint network heads, the language ID ("mr") must be passed explicitly during decoding.

⚠️ Note on Tokenizer Choice

Bhili does not currently have a dedicated tokenizer in the IndicConformer model's supported language set. We use the Marathi (mr) tokenizer as the closest alternative, since both languages use the Devanagari script.

⚠️ Limitations

1. 1.This model was predominantly trained on agricultural-domain data. As a result, performance on speech from domains outside agriculture may vary and may not be representative of the performance observed on agricultural-domain speech.

1. 1.This model was trained on audio bandlimited to 8kHz (telephony bandwidth) and resampled to 16kHz. For best results, inference audio should match this preprocessing. If feeding clean 16kHz studio audio, performance will be lower than expected.

Troubleshooting