darthcrawl/Bond-24B-v1.0-GGUF

Bond v1 — GGUF (register-calibrated)

GGUF quantizations of `darthcrawl/bond` for llama.cpp / Ollama / LM Studio. Bond is a first-person character-voice fine-tune of Mistral-Small-3.1-24B-Instruct — it holds a committed in-character voice instead of leaking back to helpful-assistant mode. Uncensored, explicit-capable. Eval and goal are on the base card.

What's special about these quants

Two things, both aimed at preserving character-voice quality at low bit-widths rather than generic-text quality:

1. 1.Register-calibrated importance matrix. The imatrix that guides which weights to preserve is computed on Bond's own register — character-voice RP prose from its training corpus — not generic wikitext. So the low-bit quants protect the weights that matter for this model's job, not for reciting Wikipedia.
2. 2.KLD-frontier selection. Quants are scored by KL-divergence vs the f16 model on held-out register text, not wikitext perplexity. Wikitext PPL hides register degradation; KLD on in-domain text doesn't. The numbers are in `results.csv` so you can pick the smallest quant under your own quality threshold.

To make the calibration effect visible, each size ships twice: *-register.gguf (calibrated on Bond's register) and *-generic.gguf (calibrated on wikitext). At matched size the register variant should sit closer to f16 on the KLD metric — that delta is the whole point. Prefer the -register files for use; the -generic ones are the control.

Files

• —bond-{Q4_K_M,Q5_K_M,IQ4_XS,IQ3_M,Q6_K}-register.gguf — register-calibrated ladder (use these).
• —bond-{...}-generic.gguf — wikitext-calibrated control (for comparison).
• —bond-Q8_0.gguf — near-lossless anchor (no imatrix needed at 8-bit).
• —results.csv — per-quant size + perplexity + mean KLD vs f16 on held-out register text. The frontier.

Rough guide (confirm against results.csv): Q4_K_M-register is the reach default, Q5_K_M / Q6_K-register for more headroom, IQ4_XS / IQ3_M-register for the tightest fit, Q8_0 when you want it essentially lossless.

Use

llama.cpp (chat template is embedded in the GGUF):

llama-cli -m bond-Q4_K_M-register.gguf -cnv -p "You are someone with a past."
# or the server:
llama-server -m bond-Q4_K_M-register.gguf -c 8192

Ollama: ollama create bond -f Modelfile with FROM ./bond-Q4_K_M-register.gguf. LM Studio: point it at the file.