pipecat-ai/smart-turn-v2

Smart Turn v2

Smart Turn v2 is an open‑source semantic Voice Activity Detection (VAD) model that tells you _whether a speaker has finished their turn_ by analysing the raw waveform, not the transcript. Compared with v1 it is:

• —Multilingual – 14 languages (EN, FR, DE, ES, PT, ZH, JA, HI, IT, KO, NL, PL, RU, TR).
• —6 × smaller – ≈ 360 MB vs. 2.3 GB.
• —3 × faster – ≈ 12 ms to analyse 8 s of audio on an NVIDIA L40S.

Links

• —Blog post: Smart Turn v2
• —GitHub repo with training and inference code

Intended use & task

The model outputs a single probability; values ≥ 0.5 indicate the speaker has completed their utterance.

Model architecture

• —Backbone : wav2vec2 encoder
• —Head     : shallow linear classifier
• —Params   : 94.8 M (float32)
• —Checkpoint: 360 MB Safetensors (compressed) The wav2vec2 + linear configuration out‑performed LSTM and deeper transformer variants during ablation studies.

Training data

• —Sentences were cleaned with Gemini 2.5 Flash to remove ungrammatical, controversial or written‑only text.
• —Filler‑word lists per language (e.g., “um”, “えーと”) built with Claude & GPT‑o3 and injected near sentence ends to teach the model about interrupted speech.

All audio/text pairs are released on the pipecat‑ai/datasets hub.

Evaluation & performance

Accuracy on unseen synthetic test set (50 % complete / 50 % incomplete)

Human English benchmark (`human_5_all`) : 99 % accuracy.

Inference latency for 8 s audio

oai_citation:7‡Daily

How to use

Please see the blog post and GitHub repo for more information on using the model, either standalone or with Pipecat.