Modelpublic

PaoAI/GLM-5.3-Flash-PaoAI-ROCmFP4-STRIX-MTP-GGUF

sourceHugging Faceupdated 13d agoView on Hugging Face

0likes345downloads

Model Card

GLM-5.3-Flash-PaoAI-ROCmFP4-STRIX-MTP-GGUF

The pure-FP4 pool quant of GLM-5.3-Flash for AMD Strix Halo fleets. The whole model on the chip's native 4-bit format (159.2 GiB), designed to be split across two Strix Halo boxes over a direct link when one box isn't enough. The MTP draft head is preserved in the file — but note: on a 2-box pool, speculative decoding is OFF (pool round-trips eat the draft's gains; measured in our fleet).

Brand: PaoAI · community quant, not affiliated with Z.AI.

Who is this for?

You have two or more Strix Halo boxes connected directly (USB4/10GbE) and want the full GLM-5.3-Flash in native FP4 split across them. If you have ONE box, use our STRIX-BALANCED instead — a pool of this model loses to a solo smaller quant on both speed and simplicity.

The recipe

What	Format
Everything (attention, experts, draft head)	Q4_0_ROCMFP4_FAST (native FP4, 4.26 bpw)
Router, norms, glue	FP32 (keeper list)
MTP draft head (blk.45)	preserved (baked into FP4)

Measured on real hardware

Metric	Result
File size	159.2 GiB (single file)
2-box pool decode	~5.4 t/s (stable ×3, both boxes ~80GB each)
Solo on one box	❌ will wedge the gfx1151 driver (98GB+ allocation wall — see BALANCED/HYBRID for solo options)
Quality	not battery-scored (pool-only use case; solo score would not represent pool serving)

Build source (required)

Both binaries need the glm5next arch and the FP4 type. Both live in [guevae2/ROCmFPX](https://github.com/guevae2/ROCmFPX), branch glm5next, commit `3345156` (glm5next arch + FP4 type merged; head of that branch — also buildable from kingjones30/ROCmFPX main once glm5next is upstreamed). Build with GGML_VULKAN=ON on Strix Halo (gfx1151).

How to run (2-box pool, direct link)

bash

# halo-2 (worker):
ggml-rpc-server --host 0.0.0.0 --port 50052
# BG1 (head):
llama-server -m GLM-5.3-Flash-PaoAI-ROCmFP4-STRIX-MTP.gguf \
  --host 0.0.0.0 --port 8081 -ngl 99 -c 32768 --parallel 1 \
  --rpc <worker-ip>:50052 -fa on

⚠️ Pool endpoint rule (learned the hard way): use the DIRECT box-to-box link addresses, never routes through a third machine — RPC traffic via a middle box deadlocks after load (head stuck in send, worker in recv).

Honest position

This is the largest and least convenient of our three models: it needs two boxes, delivers ~5.4 t/s, and can't run solo. We publish it because it completes the recipe set for fleet owners — and because the measurement behind it (pools reward lean bytes-per-token over small file size) is one of the core findings of our work.

Credits & license

—Z.AI — GLM-5.3-Flash base model (MIT license; included in repo)
—Unsloth — BF16 source + imatrix
—kingjones30 / charlie12345 — ROCmFPX format & recipes
—ggml-org — llama.cpp
—PaoAI — glm5next port, FP4 recipe, fleet benchmarking

MIT, same as the base model. Not affiliated with Z.AI.