PrathamGhaywat/s1-mini-IQ3-variants-GGUF

s1-mini IQ3 GGUF Variants

Quantized GGUFs of **superwhisper/s1-mini-GGUF** — S1-mini (0.6B text normalizer built on Qwen3-0.6B, 28 layers, 1024 hidden, 3072 FFN, 16 heads / 8 KV heads, RoPE 1M, ctx 40960) — derived from s1-mini-f16.gguf (1.44 GB, F16) in superwhisper/s1-mini-GGUF. Built with llama.cpp b10778 (llama-quantize + llama-imatrix) using importance-matrix calibrated on wikitext-2-raw-v1 (2000 paragraphs, ~1 MB, imatrix.dat 1.1 MB, 20 chunks, c512/b512).

IQ1_S / IQ1_M / IQ2_* at 1.5-2.7 bpw were unusable for this 752M model (garbage output, verified on llama.cpp b10612). All IQ3 variants at 3.0-3.7 bpw are functional and included here.

Variants

All quants use q2_K for token_embd/attn_v, q5_K/q6_K for output.weight (mixed per-layer as per llama.cpp i-quant recipe).

Calibration

• —Source: Salesforce/wikitext wikitext-2-raw-v1 train, 2000 paragraphs filtered len>20
• —Tokenized 20 chunks, c512, batch 512, PPL ~33.67
• —llama-imatrix -m s1-mini-f16.gguf -f wikitext-train.txt -o imatrix.dat --chunks 20
• —Re-quantize with: llama-quantize --imatrix imatrix.dat s1-mini-f16.gguf s1-mini-IQ3_XXS.gguf IQ3_XXS

For higher quality regenerate imatrix with --chunks 100-256 (recommended 100+ for production).

Usage - llama.cpp

For exact usage guidelines, view the orignal model card, since it will not respond properly if not ingested the data correctly: `superwhisper/s1-mini-GGUF`

Single-line (avoid PowerShell line-continuation pitfalls: must be last char with no trailing space):

llama cli -m .\s1-mini-IQ3_XS.gguf --jinja --chat-template-kwargs "{\"enable_thinking\":false}" --temp 0 -sys "You are a text normalizer for speech-to-text transcripts. The input begins with a control line specifying the styling, structure, and context settings; clean the transcript to match those settings and output only the cleaned text." -p "[Styling: semi-formal] [Structure: prose] [Context: general] so um i need to like send the the report by uh friday no wait make that thursday"

Multi-line (space before backtick):

llama cli `
  -m .\s1-mini-IQ3_XS.gguf `
  --jinja `
  --chat-template-kwargs "{\"enable_thinking\":false}" `
  --temp 0 `
  -sys "You are a text normalizer for speech-to-text transcripts. The input begins with a control line specifying the styling, structure, and context settings; clean the transcript to match those settings and output only the cleaned text." `
  -p "[Styling: semi-formal] [Structure: prose] [Context: general] so um i need to like send the the report by uh friday no wait make that thursday"

Tested on llama.cpp 0.2.0-dev b10612 / b10778. Use --temp 0 for deterministic normalization.

What did not work

IQ1_S (1.56 bpw, 241 MiB), IQ1_M (1.75 bpw, 249 MiB), IQ2_XXS (2.06 bpw, 262 MiB) produced garbage on the same prompt - expected for sub-3bpw on a 752M model.

Repository

• —Base: `superwhisper/s1-mini-GGUF` s1-mini-f16.gguf (GGUF V3, 311 tensors, 35 KV)
• —Quant toolchain: llama.cpp b10778 (llama-imatrix + llama-quantize --imatrix)
• —Author: PrathamGhaywat
• —LFS: *.gguf tracked

License

Apache 2.0 + naming clause (inherited from superwhisper/s1-mini-GGUF / Qwen3-0.6B) — model must retain the name "S1-mini" by "Superwhisper" wherever used. See LICENSE and NOTICE in the base repo.