victor/nemotron-3-5-asr-streaming

Nemotron 3.5 ASR Streaming

Gradio Space for NVIDIA's multilingual cache-aware ASR model: `nvidia/nemotron-3.5-asr-streaming-0.6b`.

Live Space: https://huggingface.co/spaces/victor/nemotron-3-5-asr-streaming

The demo exposes the model card's runtime controls:

• —target_lang, including auto language detection.
• —Cache-aware streaming chunk profiles from 80 ms through 1.12 s.
• —Optional language-tag stripping.
• —Microphone or uploaded audio input.
• —Final transcript, run details, and measured throughput.

Hardware

The live Space runs on zero-a10g with the transcribe endpoint wrapped in @spaces.GPU(duration=...). The app keeps the model on CPU between requests and moves it to CUDA inside the decorated function, which matches the working pattern used by NVIDIA's Parakeet ZeroGPU Spaces.

The critical ZeroGPU compatibility settings are applied before importing NeMo or torch:

• —NUMBA_DISABLE_CUDA=1, because early numba CUDA initialization can break ZeroGPU allocation.
• —OMP_NUM_THREADS sanitization for container values such as CPU millicores.
• —cuda-python>=13 in requirements.txt, matching NVIDIA's Parakeet requirements.

Measured with an authenticated API call on ZeroGPU, sample1.flac (13.7 s) completed on cuda in 19.4 s, or about 0.71x realtime. CPU fallback previously took 96.6 s on the same sample.

To keep or restore ZeroGPU:

hf spaces settings victor/nemotron-3-5-asr-streaming --hardware zero-a10g

To switch to the slower CPU fallback:

hf spaces settings victor/nemotron-3-5-asr-streaming --hardware cpu-basic

Deploy

hf upload victor/nemotron-3-5-asr-streaming . . --type space \
  --exclude '.git/*' \
  --exclude '__pycache__/*' \
  --exclude '.venv/*' \
  --exclude '*.pyc' \
  --commit-message 'Add Nemotron 3.5 ASR streaming demo'

hf spaces logs victor/nemotron-3-5-asr-streaming --build --follow

After it is running, verify the API:

from gradio_client import Client, handle_file

client = Client("victor/nemotron-3-5-asr-streaming", httpx_kwargs={"timeout": 600})
print(client.view_api())
print(
    client.predict(
        handle_file("sample.wav"),
        "auto",
        "Balanced - 320 ms",
        False,
        api_name="/transcribe",
    )
)

Notes

The app follows NVIDIA NeMo's cache-aware streaming inference path rather than a generic Transformers pipeline. The model card documents target_lang, att_context_size, and strip_lang_tags; those are first-class controls in this demo.

ZeroGPU startup depends on avoiding CUDA initialization before the decorated function runs. If future NeMo changes regress this, compare against the current NVIDIA Parakeet Space startup code before changing the request path.