pltobing/streaming-asr-nmt_nemo-translategemma_onnx-gguf

🎙️ Nemotron-TranslateGemma Cache-Aware Streaming ASR-NMT - ONNX-GGUF - Hugging Face Space

Real-time speech recognition with incremental transcription and live performance metrics

License: CC-BY-NC-ND 4.0 Author: Patrick Lumbantobing, Vertox-AI Copyright (c) 2026 Vertox-AI. All rights reserved.

✨ Features

✅ Live Microphone Input - Real-time speech recognition ✅ File Upload Support - Process audio files (WAV) ✅ Incremental Transcripts - Updates as text is recognized (not batch) ✅ Incremental Translation - Updates as text is translated (not batch) ✅ Real-Time Metrics - See detailed pipeline performance ✅ No Scrolling - Everything visible on standard displays ✅ Responsive Design - Works on desktop and tablet ✅ Production Ready* - Error handling, resource cleanup

🎯 What Is This?

This is a Gradio web interface for Nemotron Cache-Aware Streaming ASR - a state-of-the-art streaming speech recognition system that:

• —Performs real-time inference using a Conformer-RNN-T architecture
• —Implements cache-aware streaming for efficient incremental inference
• —Maintains a multi-threaded pipeline for low-latency processing
• —Outputs incremental transcripts (piece-by-piece as recognized)
• —Provides comprehensive performance metrics for debugging and optimization

🚀 Quick Start

Try It Now

Simply use the interface:

1. 1.For Microphone:
2. 2.Click "🎤 Mic" tab
3. 3.Adjust recording duration (5-60 seconds)
4. 4.Click "🎙️ Record"
5. 5.Speak clearly into your microphone
6. 6.Watch the transcript update in real-time

1. 1.For File Upload:
2. 2.Click "📁 File" tab
3. 3.Upload an audio file (WAV)
4. 4.Click "📁 Process"
5. 5.Watch transcription with live metrics

📊 Understanding the Metrics

The metrics display shows real-time pipeline performance:

Stage                       Data           Elapsed Time       Latency (avg)

Audio Input
Features Extraction
Features Chunking
Encoder Passes
Decoder
NMT
Transcript GUI
Translation GUI

Key Metrics Explained

Audio Input:

• —Number of audio chunks captured
• —Duration of audio processed
• —Latency per audio chunk

Feature Extraction:

• —Raw features extracted from audio
• —Feature chunks assembled for encoding
• —Latency per feature extraction step

Encoding:

• —Number of encoder inference passes
• —Time spent in encoder
• —Latency per encoder pass

Decoding:

• —Transcripts generated (actual words)
• —Blank frames (silence/filler)
• —Latency per decoding step

Translation:

• —Transcripts segmented
• —Translations generated (actual words)
• —Blank frames (silence/filler)
• —Latency per decoding step

Performance:

• —RTF (Real-Time Factor): >1.0 is faster than real-time (good!)
• —Throughput: Feature chunks processed per second
• —E2E Latency: End-to-end latency per chunk

Queue Status:

• —Monitor each processing stage's queue depth
• —Helps identify bottlenecks

🏗️ System Architecture

User Input (Mic or File)
         ↓
┌─────────────────────────────────┐
│ Nemotron Streaming Pipeline     │
├─────────────────────────────────┤
│ 1. Audio Thread (capture/load)  │ → Audio Queue
├─────────────────────────────────┤
│ 2. Preprocess Thread (features) │ → Preprocess Queue
├─────────────────────────────────┤
│ 3. Encoder Thread (Conformer)   │ → Encoder Queue
├─────────────────────────────────┤
│ 4. Decoder Thread (RNN-T)       │ → Decoder Queue
├─────────────────────────────────┤
│ 5. NMT Thread (Gemma)           │ → NMT Queue
└─────────────────────────────────┘
         ↓
   GUI Output
   ├─ Transcript (incremental)
   ├─ Translation (incremental)
   ├─ Metrics (live)
   └─ Status (current stage)

🎤 Microphone Tips

• —Environment: Use quiet environment for best results
• —Duration: 5-60 seconds configurable
• —Microphone: Any standard microphone works (built-in OK)
• —Volume: Speak at normal conversation volume
• —Distance: 6-12 inches from microphone
• —Accents: Works best with clear pronunciation

📁 File Upload Tips

Supported Formats:

• —WAV (16kHz mono recommended)

File Requirements:

• —Maximum size: Limited by server (typically 100MB+)
• —Duration: No hard limit (longer = slower)
• —Mono or stereo (stereo auto-converted to mono)
• —Any sample rate (auto-resampled to 16kHz)

Best Results:

• —Clear speech without background noise
• —Mono, 16kHz WAV files for fastest processing
• —Professional audio recordings

⚡ Performance

Typical Performance (on standard hardware):

• —Latency: 100-200ms per audio chunk
• —Throughput: 2-5 feature chunks/sec
• —Real-Time Factor: 0.8-1.2x (varies by hardware)
• —Memory: Stable during processing
• —CPU: 40-80% utilization

Factors Affecting Performance:

• —Server hardware (CPU cores, RAM)
• —Audio chunk size (10-20ms typical)
• —ASR Model size (560ms latency model used here)
• —NMT Model size (Q8\_0 model used here, ~10 token/s on m8a.xlarge CPU)
• —Concurrent users

🎨 Layout Features

✓ Compact Design - No scrolling needed ✓ Responsive Layout - Adapts to screen size ✓ Real-Time Updates - 100ms refresh rate ✓ Mobile Friendly - Works on tablets ✓ Dark Mode Support - Gradio theme support

🔒 Privacy & Security

• —Local Processing: Audio stays on your device during recording
• —Server Processing: Server receives audio for inference only
• —No Storage: Audio not saved after processing
• —No Tracking: No analytics or tracking
• —Open Source: Code is transparent and auditable

📜 License Information

License: CC-BY-NC-ND 4.0 (Creative Commons Attribution-NonCommercial-NoDerivatives 4.0)

What You Can Do ✓

• —Use for non-commercial purposes
• —Run the application
• —Study the code
• —Reference in your work (with attribution)
• —Share with others

What You Cannot Do ✗

• —Use for commercial purposes without permission
• —Modify or create derivatives
• —Sell or monetize
• —Remove copyright notices
• —Use without attribution

Attribution Required

When using this software, please include:

Based on Nemotron Streaming ASR - Gradio Demo
by Patrick Lumbantobing (Vertox-AI)
Licensed under CC-BY-NC-ND 4.0
https://creativecommons.org/licenses/by-nc-nd/4.0/

For commercial use: Contact Patrick Lumbantobing (Vertox-AI) for licensing options.

🛠️ Technical Details

Model Information

• —Model: Nemotron (Cache-Aware Streaming ASR)
• —Architecture: Conformer-RNN-T
• —Format: ONNX (portable, fast)
• —Language: English
• —Sample Rate: 16kHz mono
• —Latency Profile: 560ms+600ms
• —Links: NVIDIA model HF-Repo, ONNX model HF-repo, Google model HF-repo, GGUF model HF-repo

Pipeline Components

• —Audio Input: Microphone (sounddevice) or File (byte read)
• —Feature Extraction: Mel-spectrogram preprocessing
• —Encoder: Conformer-based acoustic model
• —Decoder: RNN-T text generator
• —Threading: 4-thread multi-threaded pipeline

Dependencies

• —Python 3.8+
• —Gradio 5.49.1
• —ONNX Runtime
• —llama-cpp-python
• —NumPy
• —soxr

📊 Benchmarks

Nemotron 560ms Model:

• —Word Error Rate (WER): ~7% (comparable to Whisper)
• —Latency: 560ms end-to-end
• —Real-Time Factor: 1.0-1.25x on standard hardware
• —Model Size: ~2.4GB ONNX

TranslateGemma Q8\_0 Model:

• —Speed: ~10 token/s on m8a.xlarge CPU
• —Real-Time Factor: 1.0-1.16x on standard hardware
• —Model Size: ~4.2GB ONNX

🐛 Troubleshooting

📚 Documentation

• —Code: app.py - Main Gradio application
• —License: LICENSE_HEADER.md - Full license information
• —Source: TBD on GitHub (link to be added)

🤝 Citation

If you use this in research, please cite:

@software{pltobing2026nemotron,
  title={Nemotron-TranslateGemma Cache-Aware Streaming ASR-NMT - ONNX-GGUF - Gradio Demo},
  author={Tobing, Patrick Lumban},
  organization={Vertox-AI},
  year={2026},
  license={CC-BY-NC-ND 4.0}
}

📞 Contact & Support

Author: Patrick Lumbantobing Organization: Vertox-AI License Questions: See LICENSE_HEADER.md Commercial Licensing: Contact Vertox-AI

🙏 Acknowledgments

• —Nemotron ASR - NVIDIA NeMo project
• —TranslateGemma NMT - Google Gemma models
• —Gradio - Hugging Face
• —ONNX Runtime - Microsoft
• —ONNX ASR - istupakov
• —Lllama CPP - llama\_cpp

📝 Version

Version: 1.1 Release Date: February 11, 2026 License: CC-BY-NC-ND 4.0 Status: Production Ready

Enjoy real-time speech recognition and speech translation! 🎤✨

For the best experience, use a modern web browser (Chrome, Firefox, Safari, Edge)