stephvax/kyutai-stt-1b-en_fr-candle-gguf

Kyutai STT-1B (en/fr) — candle GGUF quantisations

Quantised language-model weights for `kyutai/stt-1b-en_fr-candle`, in the format candle's moshi crate loads (candle_transformers::quantized_var_builder::VarBuilder::from_gguf).

Kyutai publishes candle-quantised checkpoints for its other models (`moshiko-candle-q8`, `hibiki-1b-rs-q8`) in exactly this layout — LM in GGUF, Mimi as a separate safetensors file — but not for the STT model. These fill that gap.

What is and is not here

These replace `model.safetensors` only. You still need the other three files from the upstream repo, unchanged:

• —config.json
• —tokenizer_en_fr_audio_8000.model
• —mimi-pytorch-e351c8d8@125.safetensors (the Mimi codec — not quantised)

Drop one GGUF into the model directory as model.gguf beside those three.

Not compatible with [`moshi.cpp`](https://github.com/Codes4Fun/moshi.cpp) / ggml runtimes. Those use a different GGUF layout (general.architecture=kyutai-stt, LM and Mimi in one file, tokenizer embedded). These files carry candle's tensor names, verbatim from the safetensors, and nothing else.

Measured

200 utterances of FLEURS fr_fr (test split), decoded through the reference Rust/candle path, Apple M-series:

`q8_0` is the one to use. Its WER is the upstream's within noise (−0.06 points), for 47 % less weight and roughly twice the speed. On the CPU the difference is not an optimisation but a precondition: the f32 runs below real time (0.6x), so a streaming decoder never catches up.

q4_k is published so the comparison is reproducible, not because it is worth using: it costs +0.92 WER points and gains nothing once warm.

The global WER is dragged by a tail of hard failures rather than by uniform error — the median is 10.5 %. Six clips return empty output, and they are the same six for all three variants (FLEURS test indices 29, 31, 93, 125, 140, 166), so that belongs to the model, not to the quantisation. Excluding them, q8_0 scores 15.46 %. A handful of clips are transcribed into English or Spanish: the model is bilingual en/fr with no language pin.

How they were made

candle's own rule, applied by `quantize.rs` (the same rule as candle's tensor-tools): every rank-2 tensor whose last dimension is a multiple of the block size is quantised, everything else stays F32. 102 tensors quantised, 33 kept. Tensor names are preserved verbatim — that is what makes moshi::lm::LmModel find them.

quantize model.safetensors model.q8_0.gguf q8_0

Source checkpoint SHA-256: b9e97c53229dce728d65c76bfa892f7b563c69d671899f0ebc6518582dddec6f.

Licence and attribution

Upstream kyutai/stt-1b-en_fr-candle is CC-BY-4.0 and ungated; these derivatives are redistributed under the same terms and grant no rights the upstream licence does not. Only the numeric representation of the weights differs — no training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model; questions about training content, copyright policy or model capability belong upstream.