Modelpublic

vertox-ai/nemotron-streaming-asr-multilingual

sourceHugging Faceotherupdated 3mo agoView on Hugging Face

0likes15downloads

Model Card

Nemotron Streaming ASR

Minimal local WebSocket runtime and clients for NVIDIA Nemotron 3.5 ASR Streaming FP32 ONNX artifacts.

This package contains Python runtime code, WebSocket clients, audio frontend utilities, and validation scripts. It does not package model weights, ONNX files, debug captures, local logs, release zips, or reference repos.

Install

CPU server/client install:

bash

python -m pip install -e '.[test]'

Microphone client support:

bash

python -m pip install -e '.[mic,test]'

GPU/CUDA server support is optional:

bash

python -m pip install -e '.[gpu,test]'

Use docs/gpu_cuda.md to verify onnxruntime and CUDA provider availability before starting a CUDA server.

Model Directory Contract

The server expects --model-dir models by default. The directory is intentionally local and ignored by .gitignore; do not include it in package artifacts. Required files:

—NemoPreprocessor128.onnx
—encoder-model.onnx
—encoder-model.data
—decoder_joint-model.onnx
—tokenizer.model
—vocab.txt
—config.json
—io_contract.json

The runtime validates the ONNX IO contract, cache inputs, prompt_index, vocabulary shape, and blank token placement before inference.

Commands

Start the server on CPU or CUDA:

bash

nemotron-asr-server --model-dir models --execution-provider auto
nemotron-asr-server --model-dir models --execution-provider cpu
nemotron-asr-server --model-dir models --execution-provider cuda

Stream a WAV file:

bash

nemotron-asr-client-file --url ws://127.0.0.1:8765 --wav examples/sample.wav --language en-US --print-transcript

Run the microphone client:

bash

nemotron-asr-client-mic --url ws://127.0.0.1:8765 --language en-US --frontend-preset nemotron_safe --print-transcript

Run the local evaluation matrix when model files and a reference transcript are available:

bash

nemotron-asr-eval-matrix --wav examples/sample.wav --reference examples/sample_transcript.txt --model-dir models

Debug utilities remain under scripts/; see docs/package_contents.md.

Provider Selection

nemotron-asr-server --execution-provider auto is the default. auto uses CUDAExecutionProvider with CPU fallback when ONNX Runtime reports CUDA, otherwise it uses CPUExecutionProvider. cpu forces CPU. cuda requires CUDA and exits with a clean user-facing error if unavailable.

The selected provider list is logged at server startup and model load.

Recommended ASR Modes

Use continuous ASR input with nemotron_safe as the current stable candidate:

bash

nemotron-asr-client-mic --frontend-preset nemotron_safe --asr-input-mode continuous --print-transcript

vad_island with island_fast is available for latency experiments, but it remains experimental:

bash

nemotron-asr-client-mic --frontend-preset nemotron_safe --asr-input-mode vad_island --vad-island-send-policy island_fast --print-transcript

Silence dropping/gating is disabled by default.

EC2 Server + WSL Client Quickstart

On EC2, bind the server to loopback and use SSH port forwarding from the WSL machine:

bash

nemotron-asr-server --host 127.0.0.1 --port 8765 --model-dir models --execution-provider auto

From WSL:

bash

ssh -N -L 8765:127.0.0.1:8765 ubuntu@<ec2-public-dns>
nemotron-asr-client-mic --url ws://127.0.0.1:8765 --frontend-preset nemotron_safe --print-transcript

Do not hardcode public IP addresses in repo files; use environment-specific shell history or deployment configuration.

Privacy

Audio capture flags are opt-in because they may write private speech to disk. Avoid --capture-input-wav, --capture-raw-wav, --capture-processed-wav, --capture-received-audio-dir, and transcript logging unless the capture location and retention policy are clear.

Validation

Before claiming a release or handoff, run:

bash

python -m compileall nemotron_asr scripts realtime_test.py realtime_test_mic.py nemotron_server.py nemotron_client_file.py nemotron_client_mic.py
pytest -q
python scripts/quality_gate.py --model-dir models --coverage-min 80

Citation

BibTeX

bibtex

@software{lumbantobing2026nemotronstreamingasrmultilingual,
  author       = {Patrick Lumbantobing},
  title        = {Nemotron Streaming ASR Multilingual},
  year         = {2026},
  organization = {VertoX AI},
  url          = {https://huggingface.co/vertox-ai/nemotron-streaming-asr-multilingual},
  license      = {OpenMDW-1.1}
}

Author

Patrick Lumbantobing VertoX AI