Modelpublic

prathoshap/sushrota-sanskrit-asr

sourceHugging Faceupdated 1mo agoView on Hugging Face

19likes129downloads

Model Card

Su-śrotā — Sanskrit ASR (finetuned IndicConformer-CTC)

A Sanskrit speech-recognition model finetuned for śāstric and recitational Sanskrit (chant and prose), where general-purpose Sanskrit ASR degrades badly. It is the model behind **Vāgbodhinī** and Su-śrotā, live Sanskrit chant-practice and dictation tools.

—Base: AI4Bharat IndicConformer (EncDecHybridRNNTCTCBPEModel, ~129 M params). We use the CTC head on the Sanskrit token slice.
—Author: Prof. Prathosh A P, Indian Institute of Science, Bengaluru.

Checkpoints in this repo

File	Description
`sushrota_sanskrit_asr_v13b.nemo`	Current / deployed. Adds consented real-world ("flywheel") data on top of the curated base — far more robust on in-the-wild user recordings.
`sushrota_sanskrit_asr_v5.nemo`	Prior version, curated data only. Retained for reproducibility.

Results

The headline gain of the current model is real-world robustness. Evaluated on a leakage-free, 327-clip held-out set of in-the-wild user recordings (phones, rooms, varied speakers):

eval set	metric	v5 (curated only)	v13b (current)
in-the-wild user audio	CER	7.70%	4.36% (−43%)
in-the-wild user audio	WER	45.4%	30.4%
in-the-wild user audio	SN-WER	20.7–24.4%	10.8–13.2%

On curated/studio domains, v13b preserves the base model's quality while adding that robustness:

domain	CER	WER	SN-WER (sandhi-normalised)
Studio held-out (305 clips)	4.4%	20.2%	13.0–15.7%
Bhāgavata chant (968)	~6.0%	~46%	~22–26%
Vedānta prose (718)	~7.2%	~31%	~15–19%

Note on WER vs SN-WER: Sanskrit word boundaries are largely orthographic (sandhi fuses words), so raw WER over-penalises boundary disagreements. Roughly half of the WER is spacing; CER and SN-WER (sandhi-normalised) are the meaningful numbers.

Training Data

The deployed checkpoint (v13b) was trained on 17.4 hours / 6,438 utterances, combining curated scholar recordings with consented, quality-tiered in-the-wild data.

Source	Utterances	Hours	Description
Scholar recordings (force-aligned)	2,139	6.06	21 reciters reading Bhāgavata Purāṇa, Upaniṣad, and stotra texts; forced-aligned to reference
Recitation corpus	2,504	7.59	Pre-segmented Upaniṣad (3.0 h), Gītā / Ṛgveda (3.0 h), and additional recitation (1.6 h)
TTS-speaker augmentation	637	2.00	Studio/synthetic voices for speaker robustness
Consented flywheel (in-the-wild)	1,158	1.73	User recordings from the practice tools, clean tier, ~11 input scripts
Total	6,438	17.37

Split: ~90% curated/scholar (15.65 h) + ~10% consented in-the-wild (1.73 h).

Data pipeline & quality tiering

In-the-wild audio is collected with consent through the practice tools (users read a known reference text) and automatically quality-graded at the akṣara level against that reference. Each clip is tiered — pass / override (clean, trainable), review (partial match), or low / unclear (archived). Only the clean tiers enter training.

The auto-grader was audited for reliability: all quarantined review clips were re-decoded with the improved model and promoted back to gold only when they scored a perfect akṣara match. Just 4.3% proved to be model error — confirming the remaining ~96% are genuine reader deviations (mispronunciations, disfluencies, noise), i.e. the grader correctly quarantines real problems rather than mere model disagreements.

Consented corpus

Beyond what is trained on, the flywheel has banked ~28 hours of consented Sanskrit audio from ~1,500 speakers across 11 input scripts (Devanāgarī, Kannada, Telugu, IAST, and others) — a growing resource for future, more speaker- and script-diverse training rounds.

Usage

python

import json, numpy as np, torch, soundfile as sf
import nemo.collections.asr as na

M = na.models.EncDecHybridRNNTCTCBPEModel.restore_from("sushrota_sanskrit_asr_v13b.nemo").eval()

OFF, V, BLANK = 4096, 256, 5632          # Sanskrit token slice of the aggregate vocab
def greedy(wav):                          # wav: 16 kHz mono float32
    sig = torch.tensor(wav).unsqueeze(0); sl = torch.tensor([len(wav)])
    with torch.no_grad():
        enc, _ = M.forward(input_signal=sig, input_signal_length=sl)
        lp = M.ctc_decoder(encoder_output=enc)[0].cpu().numpy()
    cols = [BLANK] + list(range(OFF, OFF + V))
    P = lp[:, cols]; P = P - (P.max(1, keepdims=True) +
        np.log(np.exp(P - P.max(1, keepdims=True)).sum(1, keepdims=True)))  # re-log_softmax on slice
    ids = P.argmax(1)
    sub = M.tokenizer.tokenizers_dict["sa"]                 # Sanskrit SentencePiece
    out, prev = [], -1
    for i in ids:
        i = int(i)
        if i != prev and i != 0: out.append(sub.ids_to_tokens([i - 1])[0])
        prev = i
    return "".join(out).replace("▁", " ").strip()

wav, sr = sf.read("clip.wav", dtype="float32")
print(greedy(wav))

The model is an aggregate multilingual IndicConformer; decode on the Sanskrit slice as above.

Provenance & consent

Scholar recordings were contributed for the purpose of building this model. In-the-wild clips are collected only with explicit user consent and store no raw IP or personal identifiers (an anonymous per-session id only). Individual reciter names are not published.

—Vāgbodhinī (chant-practice tool + full experiment report): https://github.com/prathoshap/sushrota-sanskrit-asr
—Vāgdhenu (metre-aware Sanskrit chant TTS): `prathoshap/vagdhenu`

License

Finetuned from AI4Bharat's IndicConformer — please observe the base model's license terms.

Citation

Prathosh A P, Su-śrotā: Scholar-grade Sanskrit ASR and metre-aware chant practice, Indian Institute of Science, Bengaluru, 2026.