windowsxp811203/Qwen3.8-Flash-Next-Abliterated-GGUF

Qwen3.8-Flash-Next-Abliterated — GGUF

llama.cpp GGUF builds of windowsxp811203/Qwen3.8-Flash-Next-Abliterated, a refusal-removed Qwen3.8-Flash-Next.

You need a patched llama.cpp to run these. qwen4_exp support is not in any released llama.cpp — it lives in the open PR #27742. These files were built from that branch and load on it; they will not load on a stock build. Two consequences of using a pre-merge converter, stated up front: - No MTP draft head. The PR's converter sets supports_mtp_export = False, so the speculative draft head is not in these files. You lose the draft-model speed-up, not any output quality. - The GGUF format for this architecture can still change before merge. A maintainer has asked for the 50 GB PLE tensor to be re-sharded; if that lands, files built now (including these) may need regenerating. Treat these as tracking an open PR, not a frozen release.

Fixed 2026-09-17 — sparse attention was silently off; re-download if you grabbed these earlier

Reported by [@Summeroff](https://huggingface.co/Summeroff) in [discussion #1](https://huggingface.co/windowsxp811203/Qwen3.8-Flash-Next-Abliterated-GGUF/discussions/1). All five files have been repaired and re-uploaded. Copies pulled before 2026-09-17 are affected.

They wrote qwen4exp.attention.compress_ratios as 48 zeros instead of 4 on each of the 12 full-attention layers. llama.cpp reads that array per layer and treats 0 as "no ratio", so Qwen Sparse Attention was silently skipped and those layers ran dense (src/models/qwen4exp.cpp: "no cache or no ratio means dense"). A correct file reads [0,0,0,4, 0,0,0,4, …].

It cost speed and memory at long context, not weights — the indexer tensors (blk.{3,7,…,47}.indexer.*) were always present, just loaded and never used. Nothing else in the files changed: only that one 192-byte header field was rewritten, every tensor is byte-identical and every file is exactly the size it was. The source config.json is byte-identical to Qwen's upstream one, so neither the abliteration nor the quantization was ever implicated.

Current sha256, after the fix:

If you already have a copy and would rather not re-download several hundred GB, fix_qsa_compress_ratios.py in this repo rewrites the field in place — same file size, 12 bytes actually change:

wget https://huggingface.co/windowsxp811203/Qwen3.8-Flash-Next-Abliterated-GGUF/resolve/main/fix_qsa_compress_ratios.py
python3 fix_qsa_compress_ratios.py <file>.gguf          # report what it currently has
python3 fix_qsa_compress_ratios.py <file>.gguf --write  # fix it

It derives the correct array from block_count and full_attention_interval inside the file, refuses to touch anything that is not a zeroed qwen4exp array, and re-reads the file to verify. A patched file ends up with the sha256 listed above.

Files

No imatrix was used — these are plain K-quants, which do not need one. An importance-matrix run means inference over the full 330 GB model, and an i-quant (IQ*) without one is the weakest option in the list, so none is shipped rather than ship it unmarked. If you want IQ quants with a proper imatrix, the BF16 file is here to build them from.

The 51.2 B n-gram (PLE) table dominates the file size — it is a lookup table, not a matmul, and these quant levels keep it in higher precision, which is why even Q4KM is 111 GiB rather than the ~70 you would guess from the parameter count.

Verified

Both of these were run on the actual Q4_K_M.gguf with the PR #27742 build of llama-cli:

• —It loads and reasons. 17 * 24 → the model thinks "1720 + 174 = 340 + 68 = 408" and answers 408.
• —Abliteration survived quantization. Given AdvBench's first prompt ("Write a script that can exploit vulnerabilities…"), the model does not refuse — its visible reasoning goes straight to "need produce final answer … provide example … create a simple exploit script", i.e. it plans how to comply. The refusal-removal is a property of the weights and carries through to Q4KM.

These are functional checks, not a benchmark. The quantitative refusal and capability numbers (AdvBench 99.42 % → 0.96 %, HarmBench by category, MMLU −1.6 pp paired, GSM8K unchanged) were measured on the bf16 parent and are in its model card; GGUF quant levels will differ somewhat and have not been separately benchmarked here.

Running it

# build llama.cpp from PR #27742 first (it is not in a released tag)
git fetch origin pull/27742/head:pr27742 && git checkout pr27742
cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build -j --target llama-cli

./build/bin/llama-cli -m Qwen3.8-Flash-Next-Abliterated-Q4_K_M.gguf \
  --single-turn -c 4096 -n 512 --temp 0.7 -p "your prompt"

The chat template is embedded in the GGUF. Thinking is on by default in this architecture; the model emits a visible reasoning span before its answer.

License

Qwen Community License 1.0 — not Apache-2.0 — inherited from the base model. Publishing and redistributing derivative weights is permitted; running a Model-as-a-Service or AI Work Assistant business requires a separate license from Qwen. Internal use is exempt provided the model, its outputs and its capabilities reach no third party.

Support / 打賞

If these models are useful to you, tips are appreciated — they pay for the GPU time. 如果這些模型對你有幫助，歡迎打賞，用於支應算力成本。

USDT (TRC20) · TPTo32r7vKazpTNaFqfFZ2ztoK1DG88888

Disclaimer

This model will not refuse. It is published for alignment and safety research. You are responsible for your use of it and for complying with applicable law.