Prashant2024/asr-en-openvino-f16

English ASR (OpenVINO IR)

A Conformer-CTC-BPE acoustic model for English speech recognition, converted from NVIDIA's `stt_en_conformer_ctc_large` NeMo checkpoint (NeMo → ONNX → OpenVINO IR, FP16) for CPU inference.

Files

Blank token id (for CTC decoding) is len(vocab), i.e. one past the last entry in en.json - it is not itself listed in the file.

Model details

• —Architecture: Conformer encoder + CTC head (token classification over the vocabulary above)
• —Input: log-mel spectrogram, 80 mel bins
• —Frontend config (must match at inference time - NeMo's AudioToMelSpectrogramPreprocessor defaults):
• —Sample rate: 16 kHz, mono
• —n_fft=512, hop_length=160, win_length=400, hann window
• —Pre-emphasis coefficient: 0.97
• —Per-feature (per mel-bin) mean/std normalization after log compression
• —Outputs: per-frame logits over vocab_size + 1 (vocabulary + blank)
• —Encoder time-subsampling: ~4x (mel frames -> output frames)

Usage

import numpy as np
import openvino as ov

core = ov.Core()
model = core.read_model("en.xml")
compiled = core.compile_model(model, "CPU")
infer_request = compiled.create_infer_request()

# mel_batch: float16 [1, 80, T] log-mel spectrogram built with the frontend
# config above; mel_length: int64 [1] = T
infer_request.infer(inputs={"audio_signal": mel_batch, "length": mel_length})
logits = infer_request.get_output_tensor(0).data  # [1, T', vocab_size + 1]

ids = np.argmax(logits[0], axis=-1)

Decode with standard CTC greedy decoding: collapse consecutive duplicate ids, then drop the blank id (len(vocab)), then map remaining ids through en.json and replace ▁ with a space.

Intended use

Real-time and offline English dictation / transcription. Not evaluated for noisy, far-field, or multi-speaker audio. The original checkpoint's known limitations (e.g. degraded accuracy on old/low-fidelity recordings) carry over to this conversion, since the conversion preserves weights exactly.

Provenance

Converted from the original PyTorch/NeMo checkpoint via ONNX export, then OpenVINO's model conversion API to FP16 IR. Decoded output was verified to match the original NeMo model's output on held-out audio before conversion.