Modelpublic

jakeatx/Qwopus3.8-27B-Flash-ATX-IQ4_XS-M-GGUF

sourceHugging Faceapache-2.0updated 17d agoView on Hugging Face

2likes1.4kdownloads

Model Card

Qwopus3.8-27B-Flash ATX-IQ4_XS-M (GGUF)

The ATX-IQ4_XS-M recipe applied to Jackrong/Qwopus3.8-27B-Flash, a fine-tune of Qwen3.8-27B tuned for shorter reasoning and higher MTP acceptance in agent loops. Same architecture, same 27.32B parameters, same 866 tensors as the base model, so the recipe carries over unchanged; what is new here is a model-specific importance matrix.

The recipe was designed for a single RTX 3090 / 3090 Ti (24 GB): fit a populated 200K-token context with an 8-bit key cache and the model's own MTP speculative head, and decode faster per speculative round than the stock Q4 mixes on that card. The original build and its measurements are at jakeatx/Qwen3.8-27B-ATX-IQ4_XS-M-GGUF.

file	size	notes
`Qwopus3.8-27B-Flash-ATX-IQ4_XS-M.gguf`	14.6 GiB (15,724,533,952 bytes)	4.60 bits per weight, MTP layer included
`imatrix_qwopus_flash.gguf`	13.6 MB	importance matrix computed from this model, see below
`tensor_types_ATX-IQ4_XS-M.txt`		the per-tensor type map used by `llama-quantize`

On the name. ATX-IQ4XS-M reads as: base format IQ4XS on the bulk tensors, upgrade pattern M. In llama.cpp's vocabulary the XS in IQ4XS is part of the format's name (the 4.25 bits-per-weight super-block layout of the IQ4 codebook), not a mix size, while the S/M/L suffix on the K-quants describes how many tensors are lifted above the base format. This file lifts the same tensors Q4KM does, plus the early-layer tensors Jackrong's Q4KS lifts, so it is an M-pattern mix on an IQ4XS base: 4.60 bits per weight, between Q4KS (4.63 for this model) and Q4KM (4.92) in size.

Recipe

Built from the Qwopus safetensors converted to a BF16 GGUF with llama.cpp's converter (MTP layer kept), quantized with the per-tensor type map:

tensors	format	share of weight bytes
everything not listed below	IQ4_XS	~68%
attnoutput, ssmout, ffn_down in the layers Unsloth's tier ladder upgrades first	Q5_0	~17%
attention K/V projections, output head	Q6_K	~7%
token embedding (host side)	Q4_K	~5%
the eight attention K/V tensors Q4KM keeps at Q8_0 (V in layers 11, 27, 31, 51, 55, 59, 63; K in 31)	Q8_0
FFN down in layers 0-7 and GDN qkv in layers 0-2 (the tensors Jackrong's Q4KS lifts to Q5_K)	Q5_0	~3%
MTP draft layer (blk.64)	Q5_0
GDN alpha / beta vectors (96 tiny)	Q8_0

llama-quantize --imatrix imatrix_qwopus_flash.gguf --tensor-type-file tensor_types_ATX-IQ4_XS-M.txt \
               --token-embedding-type q4_K Qwopus3.8-27B-Flash-BF16.gguf Qwopus3.8-27B-Flash-ATX-IQ4_XS-M.gguf iq4_xs

Every tensor's type was checked against the base ATX-IQ4XS-M file after quantization: identical except the eleven early-layer tensors above, which this build carries at Q50 so that every tensor Jackrong's own mix protects stays protected here.

Importance matrix. Rather than reuse the Unsloth base-model imatrix, this build computes its own from Qwopus at Q80 precision over about 226K tokens: Bartowski's `calibrationdatav3` (mixed prose, code, multilingual, chat) plus 64K-token agentic and coding prompts and a 32K-token retrieval-analysis prompt from the production prompt set the recipe was tuned on. A fine-tune shifts activation statistics; measuring them on the model being quantized is the more faithful translation of the recipe.

Why this mix: on SM86 the fastest weight format per tensor at speculative verification widths 1-5 is IQ4XS, not the 2-3 bit codebook types (IQ3S, IQ3XXS, IQ2S are instruction-bound and slower despite fewer bytes); Q50 is about 16% cheaper than Q5K; Q8_0 is the only format near the memory roof. Extra bits go where Unsloth's tier ladder puts them (attention V/K, attention output, GDN output, FFN down).

Measured (RTX 3090 Ti, TurboQuant+ fork with SM86 kernel work, MTP-3, q8_0 K / turbo3 V)

Same prompts for both files, sampled decode at temperature 1, top-k 20, top-p 0.95, 256-token generations, two runs per cell (second run with the prompt cached). Compared against Jackrong's Q4KS, the like-for-like size class (14.7 GiB, 4.63 bpw).

context	workload	ATX-IQ4_XS-M decode tok/s	Q4_K_S decode tok/s	ratio	prefill tok/s (ATX / Q4_K_S)
16K	coding	71.9	59.7	1.20	1378 / 1298
16K	agentic	78.8	70.3	1.12	1393 / 1388
16K	RAG analysis	83.8	67.4	1.24	1385 / 1377
32K	coding	72.7	64.8	1.12	1290 / 1288
32K	agentic	77.1	65.4	1.18	1286 / 1283
32K	RAG analysis	76.7	60.2	1.27	1285 / 1281
64K	coding	66.2	55.1	1.20	1099 / 1099
64K	agentic	68.8	53.0	1.30	1095 / 1103
64K	RAG analysis	67.1	57.9	1.16	1095 / 1104
100K	coding	56.2	57.1	0.98	934 / 940
100K	agentic	57.9	55.7	1.04	933 / 940
100K	RAG analysis	52.6	48.3	1.09	931 / 938
16K mean	all three	78.1	65.8	1.19
32K mean	all three	75.5	63.5	1.19
64K mean	all three	67.4	55.3	1.22
100K mean	all three	55.6	53.7	1.04

Decode is the mean of the two runs per cell; the ratio is ATX-IQ4XS-M over Q4KS. Ready VRAM at a 110,592-token window: about 19,910 MiB (ATX-IQ4XS-M) against 19,879 MiB (Q4KS). The two files prefill at the same rate. Sampled decode swings with draft acceptance (65-80% per cell here), and a few generations stopped early on their own (the model finished its answer before 256 tokens), which makes the 100K cells the noisiest: read the picture as roughly +20% decode from 16K to 64K, narrowing to a few percent at 100K, where attention over the long cache takes a growing share of each round and the weight format matters less. Same runtime, same prompts, same session for both files; Jackrong's Q4KS was measured as downloaded.

sha256 of the GGUF: 733198976d76de46326f9d685533827551d14cdef46f03c2a4d6255f42df76a2.

Run it

Runtime with the kernel and memory work: https://github.com/JakeATX/llama-cpp-qwen-ampere, branch main. Build with -DGGML_CUDA=ON -DGGML_CUDA_FA=ON -DCMAKE_CUDA_ARCHITECTURES=86. The file also loads on stock TurboQuant+ and on mainline llama.cpp (without the turbo3 cache there), with less context headroom.

GGML_Q8_TURBO3_MMA_FUSED=1 llama-server -m Qwopus3.8-27B-Flash-ATX-IQ4_XS-M.gguf \
  -c 245760 -b 4096 -ub 1024 -t 8 -tb 8 -ngl 99 -fa on -ctk q8_0 -ctv turbo3 \
  --parallel 1 --jinja --fit off \
  --cache-prompt --cache-ram 8192 --ctx-checkpoints 24 --checkpoint-min-step 10240 \
  --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0.45 \
  --spec-draft-type-k q8_0 --spec-draft-type-v turbo3

Single-user configuration (one request at a time). The cache flags are explained on the base model's card; they keep long conversations from re-prefilling and let you edit or regenerate a turn on this partly recurrent architecture. Qwopus ships its own chat template and sampling defaults (temperature 1, top-p 0.95, top-k 20), which the GGUF carries.

Quality was not re-measured for this build; the recipe's quality evidence is on the base model's card and in its paired Terminal-Bench / SciCode comparison against Q4KM. Qwopus's own capability trade-off (shorter reasoning, lower MMLU-Pro than the base) is described on Jackrong's card.

Credits

Qwen team for Qwen3.8; Jackrong for Qwopus3.8-27B-Flash; Unsloth for the dynamic-quant tier ladder this recipe follows; Bartowski for the calibration set; TheTom for TurboQuant+.

Qwen3.8 Perian project lineage

This repository is retained in the Qwen3.8 Perian checkpoints collection. Its exact position in the lineage is: Baseline/comparator quant retained with the project checkpoints. It is not a Perian-pruned or Perian-QLoRA artifact.

The final Qwen3.8 Perian GGUF release combines three reductions and one post-training stage:

—depth: 48 to 32 transformer layers;
—routed-expert width: 384 to 288 experts per layer;
—PLE n-gram capacity: 320,001,446 to 160,000,768 rows (50%, about 25.60B parameters removed), using activation-aware bigram and frequency-ranked trigram selections validated on a document-disjoint 5M-token holdout;
—rank-32 QLoRA on 12,558 normalized traces spanning math/STEM reasoning, coding/debugging, agentic tool use, retrieval, and general multi-step reasoning. The trace mixture draws from several frontier-model families, including Fable 5, GLM 5.2, Kimi K3, Claude Opus 4.7, Qwen3.8-Max, and GPT-5.6-Sol. The final merged milestone was trained through 9,336,692 supervised assistant tokens.

Earlier checkpoints in this collection do not inherit later stages merely by being listed beside them; the stage statement above is authoritative for this artifact.