Modelpublic

killbanhar/sravaani-onnx

sourceHugging Facemitupdated 1mo agoView on Hugging Face

0likes42downloads

Model Card

SraVaani-1.0 ONNX

ONNX export of ARTPARK-IISc/SraVaani-1.0, a TDT/RNNT ASR model covering 44 Indic languages/dialects, built by SPIRE Lab + ARTPARK, IISc Bangalore (paper).

All credit for the model itself goes to the original authors. This repo only provides an ONNX conversion of the encoder + decoder_joint graphs for use with onnxruntime instead of PyTorch/TorchScript — useful for platforms where PyTorch is heavy or hard to build (e.g. Android/Termux). License follows the original: MIT.

Why this exists

The original model ships as a TorchScript checkpoint (model-asr.fp16.ts) wrapped in a transformers-compatible loader, needing torch + transformers + sentencepiece at runtime — a heavy stack on resource-constrained platforms. This repo provides the same model as two ONNX graphs (encoder.onnx, decoder_joint.onnx) runnable with just onnxruntime — smaller dependency footprint, and in testing, noticeably faster than the PyTorch CPU path (see benchmarks below).

Files

File	Size	Purpose
`encoder.onnx`	~1.77GB (fp32)	Conformer encoder: `feats[B,128,T], feature_lengths[B]` → `enc_out[B,1024,T'], enc_lengths[B]`
`decoder_joint.onnx`	~43MB	LSTM predictor + joint network, single decode step: `f[1,1024,1], tgt[1,1] int32, tlen[1] int32, h_in/c_in[1,1,640]` → `logits[1,1,1,5006], aux, h_out, c_out`
`config.json`		Model config (vocabsize, blankid, durations, etc. — same as original)
`tokenizer.model`		SentencePiece tokenizer, unchanged from original
`preproc.pt`		Mel-spectrogram frontend params (window, filterbank), unchanged from original

The graphs are fp32 (CPU has no fp16 conv kernel, so export runs the model upcast to fp32 — this roughly doubles encoder.onnx's size vs the original fp16 .ts checkpoint, trading disk space for CPU-inference correctness).

Usage

Decode is a greedy TDT loop: run the encoder once per utterance, then step the decoderjoint once per encoder frame (with duration-based skipping), same as any RNNT/TDT model. See [`onnxtranscribe.py](https://github.com/piyushparkash/sravaani-voice-termux/blob/main/onnx_transcribe.py) in the companion GitHub repo for a complete reference implementation (feature extraction + greedy decode + tokenizer, using only onnxruntime, sentencepiece, torch for the STFT frontend, and soundfile`).

python

import onnxruntime as ort

enc_sess = ort.InferenceSession("encoder.onnx", providers=["CPUExecutionProvider"])
dj_sess = ort.InferenceSession("decoder_joint.onnx", providers=["CPUExecutionProvider"])
# feats: [1, 128, T] mel-spectrogram, feature_lengths: [1]
enc_out, enc_lengths = enc_sess.run(None, {"feats": feats, "feature_lengths": feature_lengths})
# then greedy-decode by stepping dj_sess once per encoder frame - see
# onnx_transcribe.py for the full loop

Benchmarks (Android phone, CPU only, no GPU/NPU)

Tested on a mid-range Android phone (OnePlus CPH2467, arm64) via Termux, on a 4.4s audio clip:

Path	Total inference time	Realtime factor
Original PyTorch (`model.transcribe(...)`)	~11s (6s clip)	~1.8x — slower than realtime
This ONNX export, `CPUExecutionProvider`	1.35s	0.31x — faster than realtime
This ONNX export, `NnapiExecutionProvider`	2.32s	0.53x — faster than realtime, but slower than plain CPU

Note: NNAPI hardware acceleration was slower than plain CPU for this model in testing — the decoder_joint's many small per-timestep calls incur more NNAPI dispatch overhead than they save. Recommend CPUExecutionProvider.

Transcripts from both the original PyTorch model and this ONNX export were verified byte-identical on real speech, and numerically match to within 2.5e-5 max absolute difference on raw encoder output (float32 rounding noise, not a real discrepancy).

Export process

See `export_onnx.py` in the companion GitHub repo to reproduce this conversion yourself. Note: export on a normal desktop PC, not Android/Termux — Termux's PyTorch build has a libc++/libcxxabi defect that corrupts TorchScript graph attribute reading during ONNX export (unrelated to this model; a general Android/Termux PyTorch packaging issue). Plain desktop PyTorch doesn't have this problem.