ilayanambi/curry-leaves-open-wake-word-model

curry-leaves — open wake word model (6 words)

A small ONNX wake word detector that runs in realtime on CPU and listens for six wake words at once:

buddy boy · hey curry · hey assistant · hey buddy · hey clone · hey dude

One 804 KB model (207 KB quantized) detects all six and tells you which one was spoken. Trained entirely on synthetic speech (macOS say, 21 voices × 4 speaking rates) with heavy augmentation.

⚠️ No human voice was used in training. The metrics below are synthetic voices testing a model trained on synthetic voices — treat them as an upper bound and read Limitations before deploying.

The six wake words

Five of six share the "hey" prefix, so the model discriminates almost entirely on the second word. The output order above is the index order in multi.onnx.json — the logit vector is [buddy boy, hey curry, hey assistant, hey buddy, hey clone, hey dude].

How it works

Internally the model is three stages, but they are fused into one file so you never see the seams:

audio 16 kHz mono
   │
   ├─ melspectrogram         raw audio → [frames, 32] mel bins
   ├─ speech embedding        → [n, 96]  (pretrained on 1000s of hours)
   └─ classifier              → [n, 6]   logits, one per wake word
         (only this stage was trained here — 205k params)

The frozen embedding stage — pretrained on thousands of hours — maps 76 mel frames (~0.76 s) to 96 numbers that capture phonetic content while discarding speaker identity and channel noise. That pretrained front end is what makes training a usable classifier on a few thousand synthetic clips feasible at all. It comes from openWakeWord; everything is fused into wakeword_allinone.onnx, verified byte-for-byte identical to running the stages separately.

The six outputs are independent sigmoids, not a softmax: almost all audio is none of the six, and outputs forced to sum to 1 cannot express "none of the above."

Files in this repo

wakeword_allinone.onnx   the model: raw 16 kHz audio → 6 scores   3.1 MB
multi.onnx.json          thresholds + word order  ← REQUIRED, tiny
test_model.py            self-test, file scoring, and live mic
README.md                this file

The model takes raw 16 kHz mono audio and outputs one score per wake word. No feature extraction, chaining, or preprocessing on your side.

`multi.onnx.json` is never optional. An ONNX graph has nowhere to carry detection thresholds or the word order, and the model is unusable without it.

Quick start

Only onnxruntime, numpy, and soundfile are needed — no PyTorch.

pip install onnxruntime numpy soundfile

The all-in-one model takes raw audio and returns the six scores. It expects exactly 31,840 samples (1.99 s) of 16 kHz mono audio, so pad or trim to that length:

import json
import numpy as np
import onnxruntime as ort
import soundfile as sf

model = ort.InferenceSession("wakeword_allinone.onnx",
                             providers=["CPUExecutionProvider"])
meta  = json.load(open("multi.onnx.json"))
WORDS      = meta["words"]
THRESHOLDS = np.array(meta["thresholds"], dtype=np.float32)

N_SAMPLES = model.get_inputs()[0].shape[1]   # 31840
LEAD      = 5_600                             # 0.35 s of silence up front

def detect(path):
    """Return the detected wake word, or None."""
    audio, sr = sf.read(path, dtype="float32")
    assert sr == 16_000, "resample to 16 kHz first"
    if audio.ndim > 1:
        audio = audio.mean(axis=1)

    # Place the clip 0.35 s in — the offset the model was trained on.
    buf = np.zeros(N_SAMPLES, dtype=np.float32)
    room = N_SAMPLES - LEAD
    buf[LEAD : LEAD + min(len(audio), room)] = audio[:room]

    logits = model.run(None, {model.get_inputs()[0].name: buf[None]})[0][0]
    over   = logits - THRESHOLDS
    return WORDS[int(over.argmax())] if (over >= 0).any() else None

print(detect("test.wav"))   # -> "hey dude"

That is the whole thing. One model, audio in, word out.

Two things that will otherwise trip you up

1. Use the 0.35 s lead-in. The model was trained with the phrase placed ~0.35 s into the window. Centre the clip instead and it scores below threshold — a spoken word gives −39.8 instead of +25.9, and you get None with no error. This is the single most common way to think the model is broken when it is not.

2. Compare logits, not probabilities. The scores are raw logits, and the thresholds are calibrated on logits. Do not apply a sigmoid — it saturates to 1.0 for confident predictions and makes the thresholds meaningless.

Live microphone

test_model.py includes a --mic mode with a rolling buffer, smoothing, and a refractory gap so one utterance fires once:

python test_model.py --mic

It opens the mic at its native rate and resamples to 16 kHz, so a 44.1/48 kHz microphone works without extra setup.

Advanced

If you need the individual stages (the raw feature models, or a quantized classifier) — for streaming pipelines or embedded targets — they are in the source repo, which also has the full streaming detector and training code.

Performance

Held-out synthetic speech, split by source clip so augmented variants of the same recording never straddle train and validation (a random split here inflates recall by ~40 points through memorization):

Correct word chosen among the six: 99.7%

End-to-end through the streaming detector: 72/72 clips correct, 0 wrong-word attributions, 0 false fires on silence, room noise, or loud noise.

Inference: 2.3 ms per 80 ms chunk (~35× realtime) on a single CPU core. Model: 205,382 parameters.

Confusion matrix (spoken → detected, validation set)

The only meaningful cross-talk:

"hey buddy"  → buddy boy    0.6%
"hey dude"   → hey assistant 0.4%
"hey curry"  → hey dude      0.2%

Everything else is below 0.2%. Notably, hey clone and hey dude — the pair that looks most similar by raw embedding distance — show zero confusion.

Limitations

Trained only on synthetic speech. These models have never heard a human voice, a real microphone, or a real room. Expect real-world performance to be worse, possibly much worse. The numbers above are an upper bound.

Two known false triggers. Words that are complete substrings of a wake word still fire on their own:

"boy" alone produces audio nearly identical to the end of "buddy boy", and the 2-second window is mostly silence either way. Adding counter-examples and oversampling them 5× fixed buddy, clone, dude, and curry — but not these two. This is a property of the phrase choice, not a fixable bug: avoid wake words containing a common word as a complete piece.

Thresholds are calibrated on synthetic audio and will likely need tuning for your microphone and room. Raise a word's threshold in multi.onnx.json if it over-fires; lower it if it misses you.

Fine-tuning on your own voice

The synthetic→real gap is the dominant limitation, and recording ~60 positive and ~40 negative samples of your own voice (about ten minutes) closes most of it. The source repo includes the recording and training tooling.

Attribution & license

Apache 2.0.

wakeword_allinone.onnx embeds the melspectrogram and speech-embedding models from openWakeWord (Apache 2.0, © David Scripka) as its frozen feature front end — these were not trained here. The classifier fused on top is original work under the same license.

Citation

@software{curry_leaves_wakeword,
  author = {Ilaya Nambi},
  title  = {curry-leaves: open wake word model},
  year   = {2026},
  url    = {https://huggingface.co/ilayanambi/curry-leaves-open-wake-word-model}
}