ShantyCam/audiodet-ced-mini

ShantyCam AudioDet — CED-mini sound-event detector

A small, CPU-real-time sound-event detector for surveillance / edge use. It combines a frozen CED-mini audio-tagging backbone (Apache-2.0, Xiaomi CED) with a compact ShantyCam MLP classifier head that maps the backbone's 527-dim AudioSet logits to 17 alarm-relevant classes.

Runs on a Raspberry-Pi-class CPU in real time (2 s window, ~17 ms/window on one core) with no Python at inference time via `ced.cpp` GGUF inference.

Classes (17)

siren, horn, gunshot, glass, knock, loud_voice, dog_bark, baby_cry, chainsaw, drilling, smoke_alarm, speech, cat, explosion, vehicle_crash, scream, music

Per-class decision thresholds ship in each head's head_meta.json.

Architecture

raw waveform 16 kHz, 2 s window
      │
      ▼
CED-mini backbone  (ced-mini-q8.gguf, ced.cpp)   → 527 AudioSet logits/probs
      │
      ▼
StandardScaler + MLP  527 → 512 → 256 → 128 → 17  (ReLU, dropout, sigmoid)
      │
      ▼
per-class probabilities → thresholds → two-consecutive-window decision

• —Window / hop: 2 s / 1 s sliding; clip score = max_t min(p[t], p[t+1]) (temporal hysteresis kills isolated single-window spikes).
• —Head weight layout (head_weights.bin, little-endian float32): scaler.mean[527], scaler.scale[527], then per layer W[out,in] row-major
• —b[out].

Files

Which head?

v4_distill is the recommended head (knowledge-distilled, best held-out F1 on the ShantyCam domain set). v4_ced is the plain hard-label baseline. Both are the same architecture and are drop-in interchangeable.

Usage (GGUF via ced.cpp)

Build `ced.cpp`, then stream 2 s windows (16 kHz float32) through the backbone to get 527-dim logits, and apply the head:

ced-batch ced-mini-q8.gguf --threads 8   # stdin: N×[32000] f32 → stdout: N×[527] f32

Load heads/v4_distill/head_weights.bin per the layout above, apply (x - mean) / scale, the 4 linear+ReLU layers, a final sigmoid, and compare each class to its threshold in head_meta.json. See recipe/train_v4_ced.py / recipe/train_v4_distill.py for the exact reference implementation.

The GGUF backbone is also published canonically by LocalAI at `mudler/ced-gguf`; it is included here so the head + backbone version are pinned together for reproducibility.

Training data & the important caveat

The head was trained on frozen CED-mini embeddings from a mixture of public datasets and a private ShantyCam camera-domain audio set that is not released (GDPR — real people/premises). The released files are model weights only: a non-reversible transformation of pooled 527-dim logits from which no source audio or personal data can be recovered.

No training audio is redistributed in this repo. Public datasets are referenced, not re-hosted; non-commercial datasets (ESC-50, UrbanSound8K, TUT Rare, CryCeleb, FSD50K-NC subset) and any augmentation derived from them are excluded from release. The one clean, fully-synthetic positive set we generated ourselves is published separately as `ShantyCam/audiodet-synth-smoke`.

Full details: `LICENSING.md`.

Evaluation (domain held-out, indicative)

Small held-out set (89 positives / 166 negatives), deploy-simulated (2 s/1 s sliding + two-window rule). v4_distill (best grid point α=0.5, T=4): recall ≈ 81 %, false-positive rate ≈ 18 %, F1 ≈ 0.75. These numbers are noisy on a set this small and should be read as indicative, not benchmark-grade. See heads/v4_distill/report.json.

License

• —Backbone GGUF: Apache-2.0 (derivative of mispeech/ced-mini; see NOTICE).
• —ShantyCam heads, ONNX head, recipe: Apache-2.0.
• —GGUF conversion / inference tooling ced.cpp: MIT (referenced).