simonfxr/turnsense.cpp-GGUF

TurnSense.cpp GGUF

Canonical F32 and selective Q8_0 GGUF artifacts for TurnSense.cpp, a native C/C++ end-of-utterance classifier built on official upstream ggml.

These files are converted from `latishab/turnsense` at immutable revision 1ddc8f679abf3d9a42a93373b0e709f0c9d7fe63. The model architecture and weights are unchanged except for deterministic LoRA merging and, for the Q8_0 artifact, weight quantization. The production runtime has no ONNX Runtime or Python dependency.

Files

SHA256SUMS contains the same checksums in machine-readable form.

Q8_0 policy

The Q80 artifact is produced from the canonical F32 GGUF with the native `turnsensequantize tool and upstream ggmlquantizechunk`:

• —Q8_0: token_embd.weight and all 210 transformer attention/FFN projection matrices.
• —F32: all 61 RMS normalization vectors and classifier.weight.

This quantizes 211 tensors and retains 62 tensors as F32. Tensor payload drops from 513.14 MiB to 136.40 MiB, a 3.76x reduction. Repeated conversion produces a byte-identical Q8_0 file. The runtime validates this policy exactly and rejects incompatible layouts.

The dynamically quantized upstream ONNX model is not the source of this Q8_0 artifact.

Usage

Download the recommended model:

hf download simonfxr/turnsense.cpp-GGUF turnsense-q8_0.gguf \
  --local-dir models

Build and run the native runtime:

git clone --recurse-submodules https://github.com/simonfxr/turnsense.cpp.git
cd turnsense.cpp
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build -j

./build/turnsense_cli \
  --model models/turnsense-q8_0.gguf \
  --backend cpu \
  --json "Could you send that report tomorrow?"

Use --backend vulkan for the Vulkan backend. The stable C API is documented in the source repository README.

TurnSense operates on text, not audio. A voice agent should evaluate the latest punctuated STT transcript at candidate pauses and combine prob_eou with VAD, latency, and product policy.

Validation

The runtime graph uses official ggml operations and optimized backend kernels. For Q80, it verifies that every quantized `getrows and mul_mat` node remains on the selected CPU or Vulkan backend rather than unexpectedly falling back.

Measured maximum absolute probability differences on the bundled seven-case fixture suite:

Vulkan was validated on an AMD Radeon RX 7900 XTX. Backend floating-point accumulation is not expected to be bit-identical.

Prompt and labels

The runtime prepends the literal prompt prefix <|user|> and applies the embedded GPT-2 byte-level BPE tokenizer. Do not append <|im_end|>.

• —Label 0: NON_EOU
• —Label 1: EOU

Applications should normally use prob_eou with a product-specific threshold rather than treating argmax as a fixed policy.

Limitations

• —The upstream model is English-focused.
• —Predictions are sensitive to punctuation and STT transcript quality.
• —Turn-taking decisions should also incorporate VAD, timing, and application context.
• —The fixture suite validates conversion parity; it is not a broad task-quality benchmark.
• —This model is not intended for safety-critical decisions.

Provenance

• —Upstream source: <https://github.com/latishab/turnsense>
• —Upstream source revision: b40a25b4da94c961b64393752e507f59295061ad
• —Upstream model: <https://huggingface.co/latishab/turnsense>
• —Upstream model revision: 1ddc8f679abf3d9a42a93373b0e709f0c9d7fe63
• —Runtime source: <https://github.com/simonfxr/turnsense.cpp>

Full source artifact hashes and deterministic conversion instructions are in `docs/model-provenance.md`.

License

The model artifacts are distributed under the Apache License 2.0, matching the upstream TurnSense model. See LICENSE. The TurnSense.cpp runtime source is separately distributed under the MIT License.