Modelpublic

aufklarer/Smart-Turn-v3.2-CoreML

sourceHugging Facebsd-2-clauseupdated 19d agoView on Hugging Face

0likes106downloads

Model Card

Smart Turn v3.2 — CoreML

End-of-turn detection for voice agents on Apple platforms. Given the last 8 seconds of the user's speech, the model returns the probability that the user has finished their turn, so an agent can reply promptly after a real endpoint and keep listening through a mid-sentence pause. It works on the raw audio (prosody, pace, intonation), not a transcript, and covers 23 languages.

The Whisper log-mel front-end is inside the model, including the zero-mean / unit-variance waveform normalisation the upstream model was trained with. Feed 16 kHz PCM and read one probability.

Model

Property	Value
Parameters	8.0 M (Whisper-Tiny encoder + attention pooling + MLP head)
Precision	float16 encoder and head, float32 audio front-end
Input	`audio` float32 `[1, 128000]` — 16 kHz mono, most recent audio last, zeros at the front
Output	`probability` float32 `[1, 1]` — turn complete if > 0.5
Window	8 s (longer turns: keep the last 8 s)
Size	16.8 MB
Deployment target	iOS 17 / macOS 14
Languages	Arabic, Bengali, Chinese, Danish, Dutch, English, Finnish, French, German, Hindi, Indonesian, Italian, Japanese, Korean, Marathi, Norwegian, Polish, Portuguese, Russian, Spanish, Turkish, Ukrainian, Vietnamese

Files

File	Size	Description
`smart_turn.mlmodelc/`	16.8 MB	Compiled CoreML program (float16 weights, float32 front-end)
`config.json`	—	I/O contract, window size, upstream revision
`LICENSE`	—	BSD-2-Clause notice for the upstream weights

Performance

Accuracy on 1,000 clips from the upstream pipecat-ai/smart-turn-data-v3.2-test set (shard train-00000-of-00010.parquet, threshold 0.5). Latency is one 8-second window on Apple M5 Pro (macOS 26.5.2) (ONNX Runtime CPU with 2 intra-op threads, CoreML on CPU + Neural Engine). Higher accuracy / F1 is better; FPR is the share of unfinished turns wrongly cut off, FNR the share of finished turns the model kept waiting on.

Model	Accuracy	Precision	Recall	F1	FPR	FNR	Latency (mean)	Latency (p95)
Upstream `smart-turn-v3.2-gpu.onnx` (mel input)	92.90%	0.912	0.944	0.927	8.48%	5.61%	—	—
`smart-turn-v3.2.onnx` (our ONNX export)	92.90%	0.912	0.944	0.927	8.48%	5.61%	36.3 ms	51.4 ms
`smart_turn.mlmodelc`	92.90%	0.912	0.944	0.927	8.48%	5.61%	3.5 ms	5.4 ms

Usage

swift

import CoreML

let config = MLModelConfiguration()
config.computeUnits = .cpuAndNeuralEngine
let url = Bundle.main.url(forResource: "smart_turn", withExtension: "mlmodelc")!
let model = try MLModel(contentsOf: url, configuration: config)

// `turn` holds the user's current turn at 16 kHz; keep the last 128000 samples.
let window = try MLMultiArray(shape: [1, 128000], dataType: .float32)
let tail = turn.suffix(128000)
let offset = 128000 - tail.count
for (i, sample) in tail.enumerated() { window[offset + i] = NSNumber(value: sample) }
let output = try model.prediction(from: MLDictionaryFeatureProvider(dictionary: ["audio": window]))
let probability = output.featureValue(for: "probability")!.multiArrayValue![0].floatValue

bash

# speech-swift CLI
speech turn --audio utterance.wav

Run it after Silero VAD reports a pause, on the whole current turn (up to 8 s). If the user resumes before the agent answers, run it again on the full turn.

Source