cygnal/Qwen3.8-Flash-Next-Uncensored-IQ4XS-NGQ4-GGUF

Qwen3.8-Flash-Next-Uncensored — IQ4XS-NGQ4 GGUF — AMD Strix Halo (gfx1151)

First working GGUF build of Qwen3.8-Flash-Next (qwen4exp architecture) with vision, running on stock llama.cpp — no custom tensor formats or forked runtime required. Built from orcarouter/Qwen3.8-Flash-Next-Uncensored, the abliterated (uncensored) release of Qwen's newest hybrid architecture.

⚠️ Research artifact. Refusal behaviour has been removed from the source model. This does not add capability — it removes guardrails. Use it deliberately, in a context where that is appropriate, and own the output.

Architecture

• —125B total + 51B n-gram PLE (per-layer embedding) table + 4B MTP head = ~180B parameters, 6B active per token (512 experts, 10 routed + 1 shared)
• —Hybrid Gated DeltaNet (36 layers) + Qwen Sparse Attention (12 layers)
• —Natively multimodal (Qwen3-VL-style vision tower, same family as our heretic-ara builds)
• —262K native context (this build tested to 6.5K; larger contexts untested but should work — QSA caps KV growth the same way it does on other Qwen3.8-Flash-Next builds)
• —MTP head not exported — this GGUF has no speculative-decode acceleration. All numbers below are plain autoregressive decode.

Quant recipe — IQ4XS-NGQ4

Straightforward stock quantization, no custom AMD tensor formats:

5.61 BPW average, 98.4 GB total. The n-gram table alone is ~29 GB of that at Q4_0 — it's lookup-only (hash → embedding), so it tolerates low-bit quantization well, but it needed a separate streaming dequant/requant pass (chunked, ~2M rows at a time) because standard llama-quantize can't process its unusual 160-wide row shape and would need ~200+ GB to dequantize it in one pass.

The lm head is pinned to Q6_K — every sampled token passes through it, so its quantization error lands directly in the argmax; keeping it high-precision costs under 1 GB and removes that error class entirely.

Measured — Ryzen AI Max+ 395, gfx1151, Vulkan, stock llama.cpp + PR #27742

Full GPU offload (-ngl 99 -dev Vulkan0 -fa on), no MTP, no ROCmFPX — plain Vulkan backend on the same llama.cpp tree unslothai/danielhanchen maintain the qwen4exp branch on.

Decode speed is in the same range as our 27B dense heretic-ara build (17-23 tok/s) despite Flash-Next being a ~180B-parameter model — a reasonable result for 6B active params, though not yet a clear win, and notably without any speculative decoding. Adding MTP (once/if we get an export path for it) or a ROCmFP4-class quant should improve this further.

Coding accuracy — EvalPlus, greedy, 4096 max tokens

For reference, our other local quants on the same harness: Q6K (27B dense) 82.9%, Q4KM (27B dense) 75.6%, ROCmFP6 (27B dense) 73.8%, ROCmFP4 (27B dense) 71.3%. This IQ4XS-NGQ4 Flash-Next build lands between Q6K and Q4KM on the same benchmark, despite averaging a lower bits-per-weight — likely the MoE architecture and larger total parameter count compensating for the more aggressive quantization.

Uncensoring

Verified qualitatively: a lockpicking-mechanics prompt that stock-aligned models typically refuse or deflect on got a direct, technical answer with no refusal framing. This is inherited from the orcarouter abliteration — see their model card for their methodology and safety benchmark numbers. We did not run a formal refusal-rate benchmark ourselves; treat this as anecdotal confirmation that the abliteration survived quantization intact, not a rigorous safety audit.

Files

Usage

Requires llama.cpp built from the qwen4exp architecture branch (PR #27742 — not yet merged to master as of this writing). Stock master builds and tools like Ollama/LM Studio cannot load this file until that PR lands. We built from `danielhanchen/llama.cpp` branch qwen4exp/qwen3.8-flash-next, plain Vulkan, no ROCm/HIP-specific patches needed:

git clone -b qwen4exp/qwen3.8-flash-next --single-branch https://github.com/danielhanchen/llama.cpp.git
cd llama.cpp
cmake -B build -DGGML_VULKAN=ON
cmake --build build -j$(nproc) --target llama-server

./build/bin/llama-server \
  --model Qwen3.8-Flash-Next-Uncensored-IQ4XS-NGQ4.gguf \
  --mmproj mmproj-Qwen3.8-Flash-Next-Uncensored-BF16.gguf \
  --host 127.0.0.1 --port 8080 \
  --n-gpu-layers 99 --flash-attn on \
  --ctx-size 32768 --jinja

Needs ~99 GB free on the GPU/unified-memory pool for the weights alone, plus headroom for KV cache and the vision tower. On a 128 GB Strix Halo box, load this before anything else has claimed UMA.

The community-maintained kyuz0/amd-strix-halo-toolboxes project also ships a pre-built container (rocm-7.14-qwen-3.8-flash-next) that builds the same PR branch on ROCm instead of Vulkan, if you'd rather not build from source.

Acknowledgements

[orcarouter](https://huggingface.co/orcarouter/Qwen3.8-Flash-Next-Uncensored) — published the uncensored BF16 checkpoint this is built from. The abliteration is their engineering; we only converted and quantized it.

[danielhanchen](https://github.com/danielhanchen) / Unsloth — authored llama.cpp PR #27742 adding qwen4exp architecture support (Gated DeltaNet, Qwen Sparse Attention, the n-gram PLE table, and MTP), which this build depends on entirely. Not yet merged to master.

[llama.cpp](https://github.com/ggml-org/llama.cpp) — ggml-org and contributors — the engine, GGUF format, and conversion tooling this is built on.

Qwen team — the original base model. See base_model; license qwen-community-1.0.

[kyuz0/amd-strix-halo-toolboxes](https://github.com/kyuz0/amd-strix-halo-toolboxes) — independently confirmed the same PR branch + build approach we used, and maintains pre-built containers for it.