Modelpublic

PaoAI/GLM-5.3-Flash-PaoAI-ROCmFP4-STRIX-HYBRID-GGUF

sourceHugging Faceupdated 13d agoView on Hugging Face

0likes868downloads

Model Card

GLM-5.3-Flash-PaoAI-ROCmFP4-STRIX-HYBRID-GGUF

The maximum-precision hybrid quant of GLM-5.3-Flash for AMD Strix Halo. Finest attention in our lineup (FP4 on the chip's fast format), a near-original draft head (Q8) — built for users who want the best-thinking version and accept a larger file plus one serving accommodation.

Brand: PaoAI · community quant, not affiliated with Z.AI.

The recipe (what's inside)

Model part	What it does	Format
👁 Attention (attn, indexer)	reads your prompt, tracks context	FP4 (q4_0_rocmfp4_fast, 4.26 bpw)
🧠 Experts (FFN MoE, 132 tensors)	the knowledge	IQ2_XXS (~2.06 bpw, imatrix)
✍️ Draft head (blk.45)	MTP speculative guessing	Q8_0 — near-original
📡 Router, glue, norms	keep signals correct	FP32 (never touched)

Chain-test results (N=3 medians — Sep 2026)

Same execution-graded coding exam as all PaoAI models: deceptive-code tracing, novel state-machine implementation, thread-safe ring buffer under a 5,000-operation stress harness. Three identical runs per model; medians with spreads.

Task (what it measures)	Median of 3	Runs
Automaton — implement novel rules + exact state trace	100	40 · 100 · 100
Ring buffer — multi-threaded code under stress	100	100 · 100 · 20
Hunt — read lying code: trace it + find the planted bug	60	60 · 60 · 100

Task median: 73/100 (runs 67 · 87 · 73 — the steadiest of our builds)

Plain read: reliable on both implementation tasks (automaton 100, threading 100). Finding the specific planted bug-line in deceptive code is its blind spot — it traces the code correctly but names the wrong line.

Speed (decode, tokens/second, per phase)

Phase	t/s (3-run range)
Spec (planning)	8.8 – 9.0
Hunt (analysis)	13.1 – 13.8
Automaton (code writing)	11.7 – 15.1
Ring buffer (code writing)	8.9 – 10.2
Publish (long output)	9.3 – 11.3

Slowest of our builds — the cost of the CPU-MoE accommodation below.

Facts

Metric	Result
File size	98.3 GiB (5 shards)
Load	⚠️ requires `--n-cpu-moe 8` (see warning)
Context	tested at 64K (`-c 65536`)
Served with	MTP speculative decoding (n-max 4), flash attention, KV q8_0

Requirements — which llama.cpp build

This model uses two things stock llama.cpp doesn't have:

—the glm5next architecture (GLM-5.3-Flash)
—the Q4_0_ROCMFP4_FAST quant type (type 101 — ROCm FP4, gfx1151-native)

Both live in [guevae2/ROCmFPX](https://github.com/guevae2/ROCmFPX), branch glm5next, commit `3345156` (glm5next arch + FP4 type merged; head of that branch — also buildable from kingjones30/ROCmFPX main once glm5next is upstreamed). Tested with GGML_VULKAN=ON on Strix Halo (gfx1151).

If you build from the wrong tree you'll see exactly one of these:

Build	Error
stock llama.cpp	`unknown model architecture: 'glm5next'`
Unsloth's glm5next branch	`invalid ggml type 101. should be in [0, 43)`

How to run (measured, locked)

bash

llama-server -m GLM-5.3-Flash-PaoAI-ROCmFP4-STRIX-HYBRID-00001-of-00005.gguf \
  --host 0.0.0.0 --port 8080 -ngl 999 --n-cpu-moe 8 \
  -c 65536 --parallel 1 --spec-type draft-mtp --spec-draft-n-max 4 \
  -fa on --cache-type-k q8_0 --cache-type-v q8_0 --lazy-mode off

⚠️ Driver-wall warning (important on Strix Halo)

This model's ~98GB Vulkan allocation will wedge the gfx1151 driver if loaded full-GPU (unkillable D-state, requires reboot — we proved it three times). The --n-cpu-moe 8 flag above is required, not optional: it keeps the GPU allocation in the proven-safe ~80GB zone by hosting 8 layers' experts in system RAM.

Known limits (honest)

—Needs the CPU-MoE accommodation (above) — slower as a result
—Hunt blind spot: traces deceptive code correctly but misidentifies the planted bug's line number (median 60)
—Long thinking: budget generously for complex tasks

Serving features explained (plain words)

Flag	What it does	Cost
`--spec-type draft-mtp --spec-draft-n-max 4`	draft head guesses 4 tokens ahead, model verifies in one pass — ~2× decode (acceptance ~50%)	needs the fine draft head we kept (Q8_0)
`-fa on`	flash attention — faster prefill + decode	none
`--cache-type-k/v q8_0`	half context memory → 64K fits one box	negligible
`--reasoning-budget 1500`	caps thinking so answers always arrive	complex tasks may need more
prompt caching (default)	follow-up turns prefill in ms	none
`--n-cpu-moe 8`	required for this model — keeps the GPU allocation under the gfx1151 driver wall	~2–3 t/s decode

Decode speed drops as context fills (≈12–15 t/s short → ~6 t/s at 33K measured) — expected attention cost, stable over time. Full plain-words guide: serving-glossary.md in our repo docs.

Credits & license

—Z.AI — GLM-5.3-Flash base model (MIT license; included in repo)
—Unsloth — BF16 source + imatrix
—kingjones30 / charlie12345 — ROCmFPX format & recipes
—ggml-org — llama.cpp
—PaoAI — glm5next port, hybrid recipe, fleet benchmarking, chain test

MIT, same as the base model. Not affiliated with Z.AI.