Modelpublic

PaoAI/GLM-5.3-Flash-PaoAI-ROCmFP4-STRIX-BALANCED-GGUF

sourceHugging Faceupdated 2d agoView on Hugging Face

1likes2kdownloads

Model Card

GLM-5.3-Flash-PaoAI-ROCmFP4-STRIX-BALANCED-GGUF

The balanced GLM-5.3-Flash quant for AMD Strix Halo (Ryzen AI Max+ 395 / gfx1151). Measured, not marketed: loads on ONE Strix Halo box with zero special accommodations — the smallest file (71.8 GiB) and the no-fuss daily driver.

Brand: PaoAI · community quant, not affiliated with Z.AI.

Updated 2026-09-21 — re-measured on the latest version (paoai-strix-engine, strix-main @ 4a8440f58): decode faster at every depth (+32% at 128K), chain re-run stated honestly below, all sweep points integrity-checked. Weights unchanged.

What is this? (30 seconds)

We took the 642 GB GLM-5.3-Flash and compressed it organ by organ: the parts that need precision keep it, the parts that don't get compressed harder. The BALANCED mix stays small enough for one box with no offload flags.

The recipe (what's inside)

Model part	What it does	Format
👁 Attention (attn, indexer)	reads your prompt, tracks context	Q3_K (~3.4 bpw)
🧠 Experts (FFN MoE)	the knowledge (most of the size)	IQ1_M (~1.75 bpw, imatrix)
✍️ Draft head (blk.45)	MTP speculative guessing	Q6_K (~6.6 bpw)
📡 Router, glue, norms	keep signals correct	FP32 (never touched)

Chain-test results (N=3 medians — Sep 2026)

We run a fixed coding exam ("chain test") where tasks are graded by execution, not opinion: the model reads deceptive code and must trace it, implements a novel state machine, and writes a thread-safe ring buffer that survives a 5,000-operation stress harness. Three identical runs per model; we report medians with spreads (identical runs vary — single scores are unreliable).

Task (what it measures)	Median of 3	Runs
Automaton — implement novel rules + exact state trace	100	100 · 100 · 40
Hunt — read lying code: trace it + find the planted bug	60	60 · 100 · 60
Ring buffer — multi-threaded code under stress	20	20 · 20 · 100

Task median: 67/100 (runs 60 · 73 · 67)

Re-run 2026-09-21 on the latest version (4a8440f58): hunt 60 · automaton 40 · ringbuffer 20 — task 40 (runs 40 · 40 · 60). Hunt and ring-buffer match the published profile; automaton dropped from 100 to 40 median (3 runs, never reached 100). Cause not yet isolated — stated plainly rather than hidden.

Plain read: novel-algorithm implementation is reliable (100 median). Multi-threaded code is the known weak spot — for concurrency tasks, generate → run the tests → regenerate once if needed.

Speed (decode, tokens/second, per phase)

Phase	t/s (3-run range)
Spec (planning)	11.0 – 13.7
Hunt (analysis)	14.5 – 15.5
Automaton (code writing)	15.7 – 18.2 ← fastest sustained code output of our builds
Ring buffer (code writing)	10.2 – 12.5
Publish (long output)	10.3 – 10.7 — filled the full 12,000-token budget every run

These are short-context speeds (tasks send a few hundred tokens of prompt).

Deep-context sweep — what happens when the window fills (7.7K → 131K)

Latest version — re-measured 2026-09-21 (4a8440f58, defaults, every point integrity-checked):

Filled context	Decode t/s (n=4)	Prefill t/s	MTP accept
7.9K	14.1	133.6	0.51
16.4K	13.3	115.5	0.59
32.8K	11.1	90.2	0.57
65.6K	10.1	67.7	0.65
98.3K	9.1	57.0	0.64
131.1K	8.6	49.4	0.64

Plain read (latest): decode is faster than the previous version at every depth (+32% at 128K); prefill wins mid-range (+9-15% at 16-33K) and is slightly behind at 64-128K.

Previous version (7.7K → 128K)

The phase table above is short-context speed. This is the other question: how fast is it when the context is actually full? One server launch per arm; we filled the window in five steps and measured decode + prefill per point — 300 output tokens each, a fresh prompt every point (no prefix-cache reuse), run at draft depth 4 and depth 6.

[image]

Filled context	Decode t/s (n=4)	Decode t/s (n=6)	Prefill t/s	MTP accept (n=4)	Memory (GTT)
7.7K	13.7	12.6	134.8	0.55	82.3 GB
30.6K	11.5	10.4	100.0	0.55	82.3 GB
61.2K	10.8	9.2	82.5	0.61*	82.4 GB
91.8K	8.4	6.8	71.8	0.54	82.4 GB
122.4K	6.5	7.1	64.7	0.46	82.5 GB

One Strix Halo, nothing else serving, flash attention, q8_0 KV — measured on the previous version (ROCmFPX port build). Prefill reproduced within 1–2% across both arms. \±0.03 sampling noise.*

Plain read: the decay is clean and predictable. Per-token cost grows straight-line at +1.8 ms per 1K of context (linear within ±3% across the whole range) — no cliff, nothing unexpected biting. Memory stays flat across 16× context (82.3 → 82.5 GB GTT), so nothing is spilling. Prefill declines gently but never collapses (134.8 → 64.7 t/s). Single-pass acceptance wobbles run-to-run (±0.05), so don't over-read any one acceptance point. Even at 122K of packed context, you still get 6.5 t/s on one box.

Draft depth 4 vs 6: depth 6 is slower at short context (12.6 vs 13.7 t/s at 7.7K — deeper guesses cost more than they return), but the longer draft pays off once the window is packed: at 122K it edged ahead (7.1 vs 6.5 t/s), because fewer verify passes matter more when each pass is expensive. At the top end both arms are within single-pass sampling noise of each other — depth 4 stays the default; depth 6 is only interesting if you live at 100K+ of context.

Facts

Metric	Result
File size	71.8 GiB (single file)
Load	full GPU, no special flags needed
Context	tested at 64K (`-c 65536`); sweep run at `-c 131072`
Served with	MTP speculative decoding (n-max 4), flash attention, KV q8_0

Requirements — which llama.cpp build

Latest version (recommended): paoai-strix-engine, branch strix-main, commit 4a8440f58 — this is what the latest-version numbers below were measured on.

Previous version: the ROCmFPX route below still works and matches the previous-version numbers.

This model uses two things stock llama.cpp doesn't have:

—the glm5next architecture (GLM-5.3-Flash)
—the Q4_0_ROCMFP4_FAST quant type (type 101 — ROCm FP4, gfx1151-native)

Both live in [guevae2/ROCmFPX](https://github.com/guevae2/ROCmFPX), branch glm5next, commit `3345156` (glm5next arch + FP4 type merged; head of that branch — also buildable from kingjones30/ROCmFPX main once glm5next is upstreamed). Tested with GGML_VULKAN=ON on Strix Halo (gfx1151).

If you build from the wrong tree you'll see exactly one of these:

Build	Error
stock llama.cpp	`unknown model architecture: 'glm5next'`
Unsloth's glm5next branch	`invalid ggml type 101. should be in [0, 43)`

How to run (measured, locked)

bash

llama-server -m GLM-5.3-Flash-PaoAI-ROCmFP4-STRIX-BALANCED.gguf \
  --host 0.0.0.0 --port 8080 -ngl 999 -c 65536 --parallel 1 \
  --spec-type draft-mtp --spec-draft-n-max 4 \
  -fa on --cache-type-k q8_0 --cache-type-v q8_0 --lazy-mode off

Add --path <webui assets> if your build serves the WebUI from disk.

Known limits (honest)

—Threading code: ring-buffer implementations crash or deadlock in most chain runs (median 20). Single-threaded generation is strong. Regenerate-once guidance above.
—Long thinking: GLM-5.3-Flash reasons heavily; give it generous token budgets for complex tasks or answers get cut mid-thought
—Driver wall (Strix Halo): Vulkan single allocations near ~98GB can wedge the GPU (needs reboot) — this model at 71.8 GiB is safely under, but don't merge shards into one file

Serving features explained (plain words)

Flag	What it does	Cost
`--spec-type draft-mtp --spec-draft-n-max 4`	draft head guesses 4 tokens ahead, model verifies in one pass — ~2× decode (acceptance ~50%)	needs the fine draft head we kept (Q6_K)
`-fa on`	flash attention — faster prefill + decode	none
`--cache-type-k/v q8_0`	half context memory → 64K fits one box	negligible
`--reasoning-budget 1500`	caps thinking so answers always arrive	complex tasks may need more
prompt caching (default)	follow-up turns prefill in ms	none

Decode speed drops as context fills — now measured in detail: 13.7 t/s at 7.7K → 6.5 t/s at 122K, straight-line cost growth with no cliff (full table + chart above). Stable over time. Full plain-words guide: serving-glossary.md in our repo docs.

Update log

date	change
2026-09-21	Re-measured on latest version (`4a8440f58`): decode faster at every depth; prefill re-run with integrity checks; chain re-run stated honestly; requirements now list the paoai-strix-engine route.
2026-09-12	Initial release.

Fleet recipes

Serving configs, ring self-heal infra, and our benchmark battery: [paoai-strix-recipes](https://github.com/guevae2/paoai-strix-recipes) (GitHub).

Credits & license

—Z.AI — GLM-5.3-Flash base model (MIT license; included in repo)
—Unsloth — BF16 source + imatrix
—kingjones30 / charlie12345 — ROCmFPX format & recipes
—ggml-org — llama.cpp
—PaoAI — glm5next port, hybrid recipe, fleet benchmarking, chain test

MIT, same as the base model. Not affiliated with Z.AI.