Modelpublic

KikoCis/Qwythos-9B-Claude-Mythos-5-1M-GGUF

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face

0likes559downloads

Model Card

[image]

Qwythos-9B-Claude-Mythos-5-1M — full GGUF ladder + KLD fidelity metrics

This is NOT my model. All weights and training are by [empero-ai](https://huggingface.co/empero-ai) (Qwythos-9B-Claude-Mythos-5-1M, Apache-2.0). This is an independent GGUF repack whose value-add is the objective fidelity measurement (KLD/PPL/Top-1 vs the Q8 reference) + provenance that the existing community GGUFs don't publish. No weights were modified.

✅ Recommended files

Use case	File	Size	Top-1 vs Q8
Archival / best	Qwythos-9B-Q8_0.gguf	9.8 GB	100%
Near-lossless	Qwythos-9B-Q6_K.gguf	7.6 GB	97.4%
Balanced	Qwythos-9B-Q5KM.gguf	6.6 GB	95.7%
Best compact (imatrix)	Qwythos-9B-IQ4_XS.gguf	5.3 GB	94.1%
Smallest (lossy)	Qwythos-9B-Q3KM.gguf	4.7 GB	83.7%

Tip: IQ4_XS dominates Q4_K_M — smaller (5.3 vs 5.8 GB) and more faithful (94.1% vs 92.3% Top-1), thanks to the imatrix.

📊 Metrics — fidelity vs the Q8_0 reference

Measured with llama-perplexity --kl-divergence (68 chunks, n_ctx 512). KLD is the gold-standard quant-fidelity metric; Top-1 match = how often the quant's argmax equals the reference's.

Quant	Size	PPL vs Q8	KLD mean	KLD p50	Top-1 match
Q8_0	9.79 GB	ref	ref	ref	100.0%
Q6_K	7.56 GB	+-0.03%	0.0056	0.0016	97.4%
Q5KM	6.64 GB	+-0.01%	0.0129	0.0043	95.7%
Q4KM	5.78 GB	+0.79%	0.0356	0.0146	92.3%
IQ4_XS	5.33 GB	+1.05%	0.0230	0.0097	94.1%
Q3KM	4.74 GB	+17.80%	0.1576	0.0725	83.7%

Reference = Q8_0, not BF16. Qwythos is a hybrid `qwen35` arch (SSM + attention, like Ornith), which llama.cpp does not lower cleanly from safetensors. So the ladder is requantized from empero-ai's official Q8_0 (near-lossless) with --allow-requantize + an imatrix, and fidelity is measured against that Q80. *(Q3KM shows real degradation — 83.7% / +17.8% PPL; use IQ4XS+ for quality.)*

[image]

🚀 Run it

bash

ollama run hf.co/KikoCis/Qwythos-9B-Claude-Mythos-5-1M-GGUF:Q5_K_M
llama-server -m Qwythos-9B-Q5_K_M.gguf -c 32768

Sampling: qwen3.5-family reasoning models want temp 1.0 / top_p 0.95 — low temp causes repetition loops. For Ollama with a recent build, use RENDERER qwen3.5 + PARSER qwen3.5.

⚠️ Honest scope

This release is fidelity-gated (KLD-measured), which is the differentiator over the existing community GGUFs. It is not an independent capability benchmark — for agentic/reasoning scores of Qwythos itself, see [empero-ai's model card](https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M) (their "Mythos-5" claims are theirs, not independently re-verified here).

🔁 Provenance