eist-edinburgh/scottish-gaelic-asr

Gaelic ASR System

This system provides Scottish Gaelic Automatic Speech Recognition (ASR) with two model options, featuring Voice Activity Detection (VAD), streaming output, and automatic punctuation.

Features

• —Model Selection: Choose between PyKaldi (traditional) or faster-whisper (modern) models
• —Voice Activity Detection (VAD): Silero VAD for intelligent speech segmentation
• —Streaming Output: Real-time progressive transcription results for faster-whisper
• —Punctuation & Capitalization: Automatic punctuation and capitalization using ONNX model
• —CPU-Only: All models run on CPU (no GPU required)

Model Options

1. Faster-Whisper Large V3 Turbo Gaelic (Recommended)

• —Model ID: eist-edinburgh/whisper-large-v3-turbo-gaelic-ct2
• —Runs directly in the Gradio app (Python 3.10+)
• —Faster inference with CTranslate2 optimization
• —Includes Silero VAD for intelligent speech segmentation
• —Streaming output: See transcription progress in real-time
• —No separate server required
• —CPU-only mode (device forced to CPU)

2. PyKaldi Model

• —Traditional PyKaldi-based ASR
• —Requires separate ASR server (Python 3.7)
• —Uses XLSR features + Kaldi decoder + RNNLM rescoring
• —Can optionally apply punctuation model to output

Usage

The Gradio interface provides a dropdown menu to select between models:

• —Upload an audio file or use the microphone
• —Select your preferred model from the dropdown
• —For faster-whisper: Configure additional options:
• —Compute Type: int8 (faster, less memory) or float16 (more accurate)
• —Use Silero VAD: Enable Voice Activity Detection to segment audio by speech activity
• —Add Punctuation & Capitalization: Apply punctuation model for better readability
• —For PyKaldi: Optionally enable punctuation & capitalization
• —Click "Transcribe" to transcribe

Faster-Whisper Configuration Options

When faster-whisper is selected, you can customize:

Note:

• —All models run on CPU only (no GPU acceleration)
• —Models are loaded on-demand and cached
• —With VAD enabled, faster-whisper provides streaming output - you'll see transcription progress in real-time as each speech segment is processed

Installation

Quick Start (Main App)

pip install -r requirements.txt
python main.py

Full System (Both models with separate client)

Run both the ASR server (for PyKaldi) and Gradio client:

# Terminal 1: Start ASR server (Python 3.7)
pip install -r requirements_server.txt
python asr_server.py

# Terminal 2: Start Gradio client (Python 3.10+)
pip install -r requirements_client.txt
python main_client.py

Model Details

Faster-Whisper

• —Based on: SYSTRAN/faster-whisper
• —Model ID: eist-edinburgh/whisper-large-v3-turbo-gaelic-ct2
• —Language Code: cy (Welsh language code used for Gaelic)
• —Beam Size: 5
• —Device: CPU only (forced)
• —Loading: On-demand when first used, then cached
• —VAD: Silero VAD (enabled by default)
• —Segments audio based on speech activity
• —Each speech segment is transcribed separately
• —Results are progressively yielded (streaming output)
• —Punctuation: ONNX-based punctuation model (optional)
• —Adds proper punctuation (., ?, ,)
• —Capitalizes appropriately
• —Handles Gaelic-specific patterns (dh', Mc/Mac prefixes, etc.)

PyKaldi

• —XLSR Model: wav2vec 2.0 XLSR-300M
• —Decoder: Kaldi lattice-based decoder
• —Language Model: RNNLM rescoring
• —Segment Duration: 10 seconds
• —Punctuation: Optional post-processing with same model as faster-whisper

Punctuation Model

• —Model ID: eist-edinburgh/gaelic-punctuation-model
• —Implementation: ONNX Runtime (CPU only)
• —Features:
• —Adds punctuation marks: period (.), comma (,), question mark (?)
• —Capitalizes sentence beginnings and proper nouns
• —Handles Gaelic-specific patterns:
• —Apostrophes: dh', o'
• —Name prefixes: Mc, Mac, Mhac, Nic, Mhic
• —Compounds with hyphens
• —Special words: Ill (Island)
• —Sliding window approach for long text
• —Probability-based predictions with overlapping window fusion

Voice Activity Detection (VAD)

• —Model: Silero VAD from snakers4/silero-vad
• —Purpose: Detect speech segments in audio
• —Benefits:
• —Filters out silence and non-speech audio
• —Better transcription quality
• —Enables progressive/streaming output
• —Efficient processing of long files
• —Device: CPU only

Architecture

┌─────────────────────────────────────────────────────────────┐
│                      Gradio Interface                        │
│                   (main.py / main_client.py)                 │
└────────────┬────────────────────────────────┬────────────────┘
             │                                │
             │ faster-whisper                 │ PyKaldi
             │ (runs locally)                 │ (via HTTP)
             │                                │
             ▼                                ▼
    ┌────────────────────┐         ┌──────────────────┐
    │  Silero VAD Model  │         │   ASR Server     │
    │  (CPU, on-demand)  │         │  (asr_server.py) │
    └────────┬───────────┘         │   Python 3.7     │
             │                     └──────────────────┘
             ▼
    ┌────────────────────┐
    │ Faster-Whisper CT2 │
    │  (CPU, on-demand)  │
    └────────┬───────────┘
             │
             ▼
    ┌────────────────────┐
    │ Punctuation Model  │◄───────── (also used by PyKaldi)
    │   (ONNX, CPU)      │
    └────────────────────┘

Configuration Reference

For HuggingFace Spaces configuration, check out: https://huggingface.co/docs/hub/spaces-config-reference

Technical Notes

CPU-Only Operation

All models are configured to run on CPU:

• —Faster-whisper: device="cpu" forced
• —Silero VAD: device="cpu" forced
• —Punctuation model: device="cpu" forced, uses CPUExecutionProvider for ONNX

Streaming Output

When using faster-whisper with VAD enabled:

1. 1.Audio is segmented into speech chunks using Silero VAD
2. 2.Each chunk is saved to a temporary file
3. 3.Whisper transcribes each chunk
4. 4.Punctuation is optionally applied to each chunk
5. 5.Results are progressively yielded to the UI
6. 6.User sees transcription build up in real-time

Model Caching

Models are loaded on-demand and cached in memory:

• —MODEL_CACHE["WHISPER_MODEL"]["{device}_{compute_type}"]
• —MODEL_CACHE["VAD_MODEL"]["MODEL_{device}"]
• —MODEL_CACHE["PUNCTUATION_MODEL"]["{device}"]

Different configurations are cached separately for efficient reuse.