Modelpublic

JamieBradfield/qwen3.8-9b-hermes-fc-clean-restraint-GGUF

sourceHugging Faceapache-2.0updated 25d agoView on Hugging Face

0likes247downloads

Model Card

Qwen3.8-9B Hermes FC — Clean Restraint (GGUF)

ROCmFPX-quantized version of `JamieBradfield/qwen3.8-9b-hermes-fc-clean-restraint` (BF16 merge in the parent repo).

Quant details

file	quant	size	notes
`qwen3.8-9b-hf-fc-v29-armB2-175-Q4_0_ROCMFP4_FAST.gguf`	Q40ROCMFP4_FAST	4.69 GB	ROCmFPX (AMD RDNA3 kernels); fast-path quant of the BF16 merge

Convert/quantize: llama-rocmfpx fork, convert_hf_to_gguf.py --outtype bf16 → llama-quantize Q4_0_ROCMFP4_FAST.

ROCmFPX quants target AMD ROCm inference (RX 7700 XT in the author's rig, 12 GB VRAM, served at 64k context with q8_0/turbo3 KV). Perplexity on a held-out probe ruler: Q4 1.148 vs Q8 1.145 — this quant is effectively lossless vs the near-lossless Q8 on the ruler that matters. For portable use, convert from the BF16 merge in the parent repo instead.

Evaluation

See the parent repo card for the full 50-probe battery. Headline: contamination-free (T4 0/10 fired, 0 drift vs v28's 2/10), restraint (T3 1/10 vs 5/10), clean format (T2 10/10 fired, 10/10 format-exact), T1 6/20 — the known assistant-final stall, documented in the parent card. This is an experimental checkpoint; the continuation fix (r4) is in training.

Note on the MTP head

The quant preserves the 15-tensor MTP head from the base (mtp.* tensors, 442 total) — present in this GGUF and usable with --spec-type draft-mtp on supporting builds.