1337Hero/Qwopus3.8-27B-Flash-ROCMFP4-GGUF

Qwopus3.8-27B-Flash — ROCmFP4 GGUF (experimental, AMD RDNA4 / gfx1201)

Two experimental 4-bit quantizations of Jackrong/Qwopus3.8-27B-Flash, a 27.3B-parameter dense multimodal model (qwen35 architecture, 64 layers, 262K context, hybrid linear/full attention, bundled MTP draft layer, Qwen3.5-style vision encoder). Converted from the BF16 safetensors and quantized on a Radeon AI PRO R9700 (gfx1201, RDNA4).

[!IMPORTANT] These files do not run on upstream llama.cpp, Ollama, LM Studio, or vLLM. They use the custom Q4_0_ROCMFP4 tensor layouts and require a ROCmFPX build as described below. Unsupported runtimes should reject the files; if a tool appears to load one anyway, do not trust the output.

[!WARNING] Validated on RDNA4 gfx1201 (loads, coherent generation smoke) and benchmarked on Strix Halo gfx1151 (llama-bench + sustained MTP decode sweep). Wikitext-2 perplexity was measured against the BF16 source. No HumanEval, MBPP, or agentic evaluation has been run, and vision output quality was not evaluated. See What was not measured before relying on either file.

Which file?

Perplexity: wikitext-2-raw test set, llama-perplexity -c 2048, full 145 chunks, BF16 baseline = 6.1651 ± 0.038. The quant deltas are small — both files hold up well for 4-bit.

Throughput measured on Strix Halo (Ryzen AI Max+ 395, gfx1151, Radeon 8060S, ROCm backend, 3 reps, ± is stddev). STRIX_LEAN is 16% smaller, 12% faster at prefill and 17% faster at decode on that hardware. The same ordering held in a single-run smoke on R9700 (gfx1201): 24 t/s vs 17 t/s decode. STRIX_LEAN is the better pick when speed or size matter — and on this model its perplexity cost (+2.49%) is modest; the plain Q4_0_ROCMFP4 spends more bits on FFN-down and attention projections and is the conservative quality choice for sensitive workloads.

Both files carry the same content: the full text model, the bundled MTP draft layer (65th block), and nothing else. Vision is served separately — see below.

MTP draft layer

The source model ships one multi-token-prediction draft layer. It is bundled into both files (block count 65 = 64 trunk + 1 MTP). ROCmFPX uses it for speculative decoding automatically; you do not need a separate draft file. The draft layer follows the same quantization mix as the trunk, so a heavy quant trades draft accuracy for size the same way the trunk does.

Measured MTP serving recipe (Strix Halo, gfx1151)

Sweep on the STRIXLEAN file, single request, ~480-token prompt, 768-token streaming generation, median of 3 runs, ROCm backend, `-b 1024 -ub 512`, q40 K/V on main and draft caches:

Best measured formula: draft n-max 3, p-min 0.6, q4_0 K/V everywhere — +29% sustained decode over no-spec. The sweet spot is shallow: this finetune's draft acceptance collapses after the first two positions (0.83 / 0.38 / 0.20 at positions 1-3, then ~5%), so deeper drafting is pure overhead — n-max 6 ties n-max 4 and loses to n-max 3. If your finetune of this model line behaves differently, re-measure before trusting any n-max above 3.

# via the ROCmFPX wrapper (Strix Halo):
MODEL=Qwopus3.8-27B-Flash-MTP-BF16-to-ROCmFP4-STRIX_LEAN.gguf \
DEVICE=ROCm0 BATCH_SIZE=1024 UBATCH_SIZE=512 \
CACHE_TYPE_K=q4_0 CACHE_TYPE_V=q4_0 \
SPEC_DRAFT_N_MAX=3 SPEC_DRAFT_P_MIN=0.6 SPEC_DRAFT_P_SPLIT=0.10 \
scripts/run-rocmfpx-mtp-server.sh

Vision: mmproj-Qwopus3.8-27B-Flash-F32.gguf

The model is multimodal. The vision projector is not inside the quant files; it ships as a plain F32 GGUF in this repository (1.72 GiB, converted from the same BF16 source, not quantized). Pass it with --mmproj. A stock F32 projector was chosen deliberately: the ROCmFP4 layout buys nothing on the vision tower and F32 keeps image encoding artifact-free.

Required runtime

git clone https://github.com/charlie12345/ROCmFPX.git
cd ROCmFPX && git checkout main   # validated at commit a4e9424 (2026-08-26)

# RDNA4 build (gfx1201):
scripts/build-rdna4.sh    # -> build-rdna4/

Example deployment

./build-rdna4/bin/llama-server \
  -m Qwopus3.8-27B-Flash-MTP-BF16-to-ROCmFP4-STRIX_LEAN.gguf \
  --mmproj mmproj-Qwopus3.8-27B-Flash-F32.gguf \
  -ngl 999 \
  --jinja \
  -c 262144 \
  -ctk q8_0 -ctv q8_0

Both quants were smoke-run on a single R9700 with all layers offloaded, coherent generation observed, ~2.7 GB VRAM spare on the 32 GB card for the STRIX_LEAN file at short context. For long contexts, quantize K/V cache further or split across cards.

What was not measured

• —Task quality. Wikitext-2 perplexity was measured (see the file table); HumanEval, MBPP, and agentic gates were not.
• —Vision quality and speed. The mmproj loads and the server starts with it attached; image-grounded generation quality was not evaluated.
• —R9700 (`gfx1201`) formal throughput. Load + coherent generation smoke and the perplexity runs only; the sustained MTP sweep and llama-bench campaign were on Strix Halo gfx1151.

Artifact

Quantization commands (ROCmFPX a4e9424, build-rdna4):

./build-rdna4/bin/llama-quantize \
  Qwopus3.8-27B-Flash-MTP-BF16.gguf \
  Qwopus3.8-27B-Flash-MTP-BF16-to-ROCmFP4-STRAIGHT.gguf \
  Q4_0_ROCMFP4

./build-rdna4/bin/llama-quantize \
  Qwopus3.8-27B-Flash-MTP-BF16.gguf \
  Qwopus3.8-27B-Flash-MTP-BF16-to-ROCmFP4-STRIX_LEAN.gguf \
  Q4_0_ROCMFP4_STRIX_LEAN

Verify after download:

sha256sum -c SHA256SUMS

Limitations

• —Requires the ROCmFPX fork; no upstream llama.cpp compatibility.
• —Formal throughput measured on Strix Halo gfx1151 only (llama-bench + sustained MTP sweep, ROCm backend, build f6ed57d). RDNA4 gfx1201 has a load + coherent generation smoke (LEAN ~24 t/s, STRAIGHT ~17 t/s decode, single run, no warmup) and the perplexity runs, but no throughput campaign there.
• —The MTP recipe is measured on the STRIX_LEAN file. The STRAIGHT file was not swept; its deeper bit mix may shift draft acceptance, so n-max 3 is a starting point, not a promise, for that file.
• —Quantized with ROCmFPX a4e9424 (2026-08-26); benchmarked with the framework's f6ed57d (2026-08-05). Both tensor formats predate that fork point, but the version gap is noted for reproducibility.
• —The 262K context window was not exercised. KV cache sizing at full depth is on the user.
• —Dense 27B: the STRAIGHT file needs a 32 GB card (or a split) for the weights alone at 262K context with q8 K/V.

License and attribution

• —Base model: Qwopus3.8-27B-Flash, Apache-2.0, by Jackrong, built on the Qwen3.8-27B line from the Qwen team. This repository redistributes a converted and quantized derivative under the same license.
• —Format and execution path: the Q4_0_ROCMFP4 and Q4_0_ROCMFP4_STRIX_LEAN representations and kernels are the work of the ROCmFPX project, which builds on llama.cpp.
• —This repository: the quantized artifacts and the RDNA4 conversion.

Qwopus and Qwen are marks of their owners. This community quantization is not affiliated with or endorsed by Jackrong, the Qwen team, AMD, ROCmFPX, or llama.cpp.