pugant/Qwen3.6-35B-A3B-MTP-Q6_0_ROCMFPX

Qwen3.6-35B-A3B — Q60ROCMFPX (Strix Halo / gfx1151, Vulkan)

TL;DR

Qwen3.6-35B-A3B (35B params, 3B active per token, Qwen3.5-MoE family) quantized to `Q6_0_ROCMFPX` (~6.5 BPW) — the quality tier of the ROCmFPX fork family, which we serve and benchmark on our lab runtime (full source: pugant/strix-nebulosa; upstream: charlie12345/ROCmFPX). This is the highest-quality ROCmFPX preset for this model, targeting coding-agent workloads where quality matters more than raw decode speed. MTP (multi-token prediction) layers are included: serve with --spec-type draft-mtp --spec-draft-n-max 3 to enable speculative decoding (up to 67 tok/s on deterministic content, +37% over plain tg128).

⚠️ This GGUF is for the ROCmFPX fork of llama.cpp. It will not load in stock llama.cpp.

⚠️ Critical warnings — read before use

• —`Q6_0_ROCMFPX` is a fork-specific type. Stock llama.cpp will refuse to load it. Requires a ROCmFPX fork build — see Runtime. Upstream charlie12345/ROCmFPX loads these files too (any build with the custom GGML types).
• —Best served on the Vulkan backend (RADV). The Q6 ROCmFPX staging path is optimized in the Vulkan build of the fork: on the ROCm backend (older commits) pp512 drops ~50%. On Vulkan RADV: 1120 pp512 / 49.2 tg128.
• —AMD RDNA 3.5 (gfx1151 / Strix Halo) is the target. Tested on Radeon 8060S iGPU. Not tested elsewhere.
• —On Strix Halo, GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 (ROCm) or the Vulkan backend with default ICD is required — VRAM partition is 512 MB.

Benchmarks

Hardware: AMD Ryzen AI Max+ 395, Radeon 8060S (gfx1151, 128 GB unified LPDDR5X). Method: llama-bench -ngl 999 -fa on -p 512 -n 128, Vulkan RADV backend (fork build).

Q60ROCMFPX trades decode speed for quality: vs the fp4 preset it is ~-39% tg, but at ~6.5 BPW it sits in Q6K-class quality territory (community K-quant comparisons put Q60ROCMFPX at Q6K lossless level, with +30% pp over Q6_K on the fork's optimized paths).

MTP (speculative decoding)

With --spec-type draft-mtp --spec-draft-ngl all --spec-draft-p-min 0.0 --spec-draft-p-split 0.10 (config validated by the fork's regression guard for qwen35-a3b). n-max sweep on this file (server-timing, 2 prompts × 2 runs, ctx 16k, Vulkan RADV):

Recommendation: n-max 3 for mixed/agentic workloads (best deterministic content, balanced prose); n-max 2 for pure decode. Position-4 acceptance is only ~0.51 on this base model, so the extra draft cost of n-max 4 is not paid back. Acceptance pos-1 0.82–0.87, mean accepted length 2.5–3.7 tokens.

Comparison — every Qwen3.6-35B-A3B variant we tested

All rows: same host (Ryzen AI Max+ 395, 8060S, 128 GB), same method (llama-bench -ngl 999 -fa 1 -p 512 -n 128), 2026-08-14.

Notes:

• —"Vulkan RADV (fork)" = the charlie12345/ROCmFPX fork built Vulkan-only — the only Vulkan build that runs the ROCmFPX tensor types (stock llama.cpp rejects them).
• —⚠️ pp 520 on ROCm: the Q6 staging path is unoptimized in the ROCm build — for Q60ROCMFPX the reference backend is Vulkan.
• —tg128 leaderboard: fp4 LEAN on the Vulkan fork (81.6) > fp4 on ROCm (71.2) > UD-Q5KM on RADV (57.9) > Q6 (49.2). pp leaderboard: ROCm fp4 (1420.7) > RADV fp4 (1164.7) > Q6 RADV (1120.4). The Q6 is the quality tier, not the speed tier.
• —Our other Strix Halo ROCmFPX repos (grug-35b-v2, Ornith-1.0-35B, Nemotron-Lightning-30B) were benchmarked in ROCm containers (HIP backend) — do not compare their tg/pp numbers directly against the Vulkan rows above.

Usage (Vulkan, fork build)

llama-server -m Qwen3.6-35B-A3B-MTP-Q6_0_ROCMFPX.gguf \
  -ngl 999 -fa on --jinja --parallel 4 \
  --top-p 0.95 --top-k 20 --temperature 1.0 \
  --reasoning on --reasoning-budget 16384
# + MTP: --spec-type draft-mtp --spec-draft-ngl all --spec-draft-n-max 3

Note: without --reasoning-budget, thinking defaults to unlimited (INT32_MAX) and can run away.

Quantization details

• —Type: Q6_0_ROCMFPX — fork-specific (~6.5 BPW).
• —Source: BF16 GGUF from unsloth/Qwen3.6-35B-A3B-MTP-GGUF, quantized with llama-quantize from the fork (imatrix pass attempted; final config in repo history).
• —MTP layers retained (~0.8 GiB extra): the file can be served with or without speculative decoding; MTP is off unless --spec-type draft-mtp is passed.
• —Base model: Qwen/Qwen3.6-35B-A3B (Apache-2.0).

License

Apache-2.0 (inherited from the base model). See LICENSE.

Runtime

Benchmark environment: one bare-metal AMD Strix Halo (Ryzen AI MAX+ 395, 128 GB) — full dated configuration and measurement policy: BARE-METAL.md

Requires a ROCmFPX fork build (custom tensor types — stock llama.cpp refuses the file). Recommended: our lab build (pugant/strix-nebulosa, main) — reasoning budget and persistent prompt cache on every model; drafter features where the model ships one: see the engine section of its README.

Runs on any build with the Q60ROCMFPX type (MTP speculative decoding included in the GGUF); our lab build adds reasoning budget + persistent prompt cache.

Everything here is experimental and provided as-is, at your own risk.