q1805/hubert_large-german-IPA-v2

HuBERT Large German IPA Phoneme Scorer (v2)

Key Highlights & Performance

• —Comprehensive Multi-Domain Training: Scaled from clean audiobooks to diverse acoustic environments including parliamentary debates, crowd-sourced phone audio, and studio recordings.
• —Superior Accuracy: Reached an Evaluation Phoneme Error Rate (PER) of 22.28% (Accuracy > 77.7%) on unseen validation data, outperforming v1 across all acoustic categories.
• —Low Latency: Average inference latency of ~32 ms per utterance on NVIDIA L4 (Tensor Cores / FP16), enabling real-time scoring in production. ---

📚 Training Dataset Architecture (123 GB Mega Dataset)

The model was fine-tuned on a composite dataset `q1805/german-pronuncheck-mega-dataset`, combining three complementary sources:

• —1.German Parliamentary Debates (Bundestag Corpus): Fast, spontaneous political discourse with natural room acoustics and public address microphones.
• —2.Mozilla Common Voice (German v17): Thousands of diverse speakers recorded on consumer smartphones and PC headsets with varied regional German dialects.
• —3.Multilingual LibriSpeech (MLS German): High-fidelity studio audiobook narrations.
• —Total Samples: 1,249,116 training utterances + 86,575 validation utterances.
• —Phonemizer: Converted to German IPA via espeak-ng using a fork-safe parallel pipeline (preserve_punctuation=False, with_stress=True). ---

⚙️ System-Level Engineering & Training Parameters

• —Compute Infrastructure: Google Cloud Platform (GCP) Compute Engine VM.
• —Accelerator: 1x NVIDIA L4 Tensor Core GPU (24 GB GDDR6 VRAM, Ada Lovelace architecture).
• —Host Resources: 4 vCPUs, 16 GB RAM, 500 GB High-Throughput NVMe SSD.
• —Total Training Time: 70 hours 25 minutes (73,050 total steps, 3 full epochs).

🛠️ Hardware & Memory Optimization Techniques:

• —Mixed Precision (`fp16=True`): Enabled NVIDIA L4 Tensor Cores, cutting memory by 50% and doubling matrix throughput.
• —Gradient Checkpointing (`gradient_checkpointing=True`): Mitigated memory spikes from long audio sequences ($O(N^2)$ attention matrices).
• —Multi-Worker I/O (`dataloader_num_workers=4`): Eliminated GPU starvation by asynchronously feeding audio arrays.
• —Effective Batch Size = 32: Configured via per_device_train_batch_size = 2 and gradient_accumulation_steps = 16.
• —Memory Defragmentation: PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True.
• —Learning Rate Policy: Linear warmup (1,000 steps) to 3e-5, followed by linear decay to 0.0. ---

📈 Comprehensive Benchmark Comparisons

1. Out-of-Domain Benchmark: Mozilla Common Voice Spontaneous Speech 4.0 (sps-corpus-4.0)

Evaluated on 100% of the completely unseen Mozilla Common Voice Spontaneous Speech 4.0 German dataset (natural speech with diverse accents and background noise):

2. Multi-Domain Benchmark: Mega Dataset 10% Held-Out Test Split (86,575 Utterances from q1805/german-pronuncheck-mega-dataset)

Evaluated on the independent 10% test split extracted from `mega_dataset` (comprising 86,575 utterances across Parliamentary debates, crowdsourced Mozilla audio, and MLS studio audio that neither model touched during training):

💻 Quickstart Inference Code

import torch
import librosa
from transformers import Wav2Vec2Processor, HubertForCTC
# Load production model and processor
REPO_ID = "q1805/hubert-german-IPA-large-v2"
processor = Wav2Vec2Processor.from_pretrained(REPO_ID)
model = HubertForCTC.from_pretrained(REPO_ID).eval()
# Load 16kHz audio
audio_path = "german_speech_sample.wav"
audio, sr = librosa.load(audio_path, sr=16000)
# Forward pass
inputs = processor(audio, sampling_rate=16000, return_tensors="pt")
with torch.no_grad():
    logits = model(inputs.input_values).logits
    predicted_ids = torch.argmax(logits, dim=-1)
    ipa_transcription = processor.batch_decode(predicted_ids)[0]
print("Predicted German IPA:", ipa_transcription)