12384-0ashntoi/hey-fable-openwakeword

hey_fable — openWakeWord model

A custom openWakeWord wake-word detection model for the phrase "hey fable".

Trained on synthetic speech from two complementary TTS engines:

• —[Piper](https://github.com/rhasspy/piper-sample-generator) (en_US-libritts_r-medium, 850 speakers) — bulk timbre and prosody coverage
• —[Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) (27 expressive American + British voices) — natural, expressive read speech that Piper cannot cover

On top of the modal-voice clips we add a whisperized subset (STFT phase-randomisation of the modal clips) so the model fires on whispered "hey fable" even in noisy environments. Every clip is augmented with room impulse responses (MIT IR Survey), ESC-50 background mixing at SNRs from −10 to +15 dB, pitch shift, band-stop filtering, seven-band EQ, tanh distortion, and coloured noise, on top of Google's frozen speech-embedding backbone that ships with openwakeword.

Metrics

End-to-end streaming, 80 ms frames, 1200 held-out modal positives + 300 held-out whispered positives + 1200 held-out adversarial negatives, plus the 10.7-hour general-speech FP corpus that ships with the base openwakeword package.

• —Recall (modal) — synthetic modal-voice held-out positives from Piper + Kokoro.
• —Recall (whisper) — whisperized versions of held-out positives (STFT phase-randomisation).
• —Recall (Kokoro voices) — subset of the modal set from 27 expressive Kokoro voices.
• —Adversarial FPR — fraction of 1200 held-out adversarial clips ("hey table", "hey stable", "hey cable", "hey Google", "hey Siri", "a fable", "the stable", "vegetable", "comfortable", …) that fire. Intentionally hard: those phrases really do sound like "hey fable".
• —FP / hour — false fires per hour on the 10.7-hour openWakeWord validation corpus (mixed real-world speech, music, noise).

Quick start

import openwakeword
from openwakeword.model import Model

# Download the shared melspectrogram + embedding backbone (~4 MB, one-off)
openwakeword.utils.download_models()

model = Model(
    wakeword_models=["hey_fable.onnx"],
    inference_framework="onnx",
)

# Feed 16 kHz mono 16-bit PCM audio in 80 ms frames (1280 samples)
prediction = model.predict(frame)
if prediction["hey_fable"] > 0.9:  # recommended threshold
    print("wake!")

Files

Training data

• —Positives — 7500 training / 1700 held-out. 4000 Piper (850 speakers × 5 length-scales × 4 noise-scales × 3 noise-scale-ws), 2000 Kokoro (27 voices × 5 speaking speeds × 10 prosody prompts), 1500 whisperized (STFT phase-randomised versions of the modal clips).
• —Hard negatives — 5500 training / 1200 held-out synthetic clips of phonetically similar phrases. The phrase file is weighted (<count>\t<phrase> lines): "hey table", "hey stable", "hey cable" each get 20× oversampling; the /-eɪbəl/ rhyme family ("hey label", "hey Mabel", "hey sable", "hey Abel", "hey fabled", "hey fabric", "hey Fabio", …) gets 10×; syllable-substitution neighbours ("gay stable", "say fable", "weigh stable", "prey fable", …) get 5×; determiner + noun ("a fable", "the stable", "away table", …) get 3×; polysyllabic -able words ("comfortable", "notable", "capable", …) get 2×. Kokoro also synthesises these to give the negatives expressive prosody.
• —Background negatives — 2000 hours of pre-computed openWakeWord features from ACAV100M (speech + noise + music) via davidscripka/openwakeword_features.
• —False-positive validation — the 10.7-hour held-out corpus from the same dataset.

Data augmentation

Every positive and negative clip runs through the openWakeWord augmentation pipeline 3 rounds (22 500 augmented positives + 16 500 augmented negatives), with augmentation probabilities pushed above defaults for whisper-in-noise robustness:

• —Background mixing at SNR −10 to +15 dB — p = 0.9.
• —Room impulse responses from the MIT Environmental IR survey (270 rooms) — p = 0.75.
• —Coloured noise at SNR 10-30 dB — p = 0.5.
• —Pitch shift ±3 semitones — p = 0.35.
• —Seven-band parametric EQ ±6 dB — p = 0.35.
• —Band-stop filter — p = 0.35.
• —Tanh distortion — p = 0.30.

Model architecture

• —Input: 16 × 96 melspectrogram embeddings from the shared openWakeWord Google speech-embedding backbone (frozen, ships with openwakeword).
• —Head: fully-connected DNN, layer width 32, LayerNorm + ReLU, single sigmoid output.
• —~15 000 parameters — faster than real-time on a Raspberry Pi.

Training used openwakeword.train.Model.auto_train with a cosine LR schedule + warmup, three sequences (75 000 + 7 500 + 7 500 steps), adaptive negative-example weighting up to 8000× (raised on demand to hit target FP rate), stochastic-weight-averaging over the top-decile checkpoints, and a rebalanced batch (ACAV100M:1024 / adversarial:600 / positive:300).

Recommended threshold

0.9 — the highest-value operating point on this model:

• —99.5 % recall on modal held-out positives.
• —97.3 % recall on whispered held-out positives.
• —0.84 FP/hour on the 10.7-hour openWakeWord validation corpus.

Bumping to 0.5 buys a tiny amount of recall (99.6 % vs 99.5 %) at the cost of 1.7× more false positives per hour.

Limitations

• —English only. Trained purely on synthetic speech; real-user voices are not represented explicitly.
• —Adversarial phrases (hey table, hey cable) still fire ~16 % of the time even at the highest threshold — this is a phonetic-similarity ceiling, not a training failure. Real users don't randomly emit those phrases.
• —Small classifier head is not designed to distinguish speakers.
• —Whisper detection works because the model was trained on whisperized clips; extreme background noise (SNR ≤ −10 dB) will still miss most whispers.

License

Apache-2.0 for code. The model weights inherit the base openWakeWord backbone's CC-BY-NC-SA-4.0 because they were derived using Google's speech-embedding model under those terms.

Credit

• —dscripka/openWakeWord — training framework and shared embedding backbone.
• —rhasspy/piper-sample-generator — bulk synthetic speech.
• —hexgrad/Kokoro-82M — expressive synthetic speech (Apache-2.0, 27 English voices).
• —MIT Environmental Impulse Responses — reverberation augmentation.
• —ESC-50 — environmental background sounds.
• —Google's speech-embedding backbone via TFHub.