kingjones777/Xing4.0-29B-A4B-ROCmFP4-GGUF

Xing4.0-29B-A4B — ROCmFP4 GGUF (Strix Halo / gfx1151)

4-bit ROCmFP4 quantizations of XingChen-AGI/Xing4.0-29B-A4B, China Telecom AI's 29B-total / 4B-active MoE (formerly the TeleChat series), built for AMD Ryzen AI Max+ "Strix Halo" (gfx1151).

The single imatrix build lives in its own repo: [Xing4.0-29B-A4B-ROCmFP4-FAST-imatrix-GGUF](https://huggingface.co/kingjones777/Xing4.0-29B-A4B-ROCmFP4-FAST-imatrix-GGUF).

⛔ These files need the ROCmFPX fork — stock llama.cpp cannot load them

Two independent reasons:

1. 1.ROCmFP4 tensor types are ≥ 100, past stock llama.cpp's GGML_TYPE_COUNT. The file is rejected outright — no metadata edit fixes this.
2. 2.`xing4` is a new architecture. Xing uses mHC (hyper-connections) + MLA with n_embd_head_k=192 / n_embd_head_v=128, which no existing llama.cpp arch implements.

Build it from our fork — the arch lives on a branch, not main:

git clone -b xing4-port https://github.com/kingjones30/ROCmFPX.git
cd ROCmFPX && cmake -B build -DGGML_HIP=ON -DGPU_TARGETS=gfx1151 \
  -DGGML_HIP_GRAPHS=ON -DGGML_HIP_NO_VMM=ON -DLLAMA_CURL=OFF -DLLAMA_BUILD_WEBUI=OFF
cmake --build build -j --target llama-server llama-quantize llama-perplexity

Pinned at commit `cbc2cd5` (the port plus the hyper-connection correctness fix). ⚠️ main does not carry xing4 — you must use the xing4-port branch.

Files

BF16 reference: 8.3608 ± 0.249. Upstream HF transformers scores 8.4252 on the identical tokens (see Verification).

Which one

• —COHERENT — best quality of the three. The default pick.
• —FAST — fastest decode and smallest. +1.8% PPL over COHERENT.
• —STRIX_LEAN — not recommended for this model. It is measurably the slowest at decode and worse quality than COHERENT here. STRIX_LEAN's advantage comes from its PLE upgrade, and Xing has no PLE. Published for completeness, not because it wins.

What we serve. Our own seat runs the COHERENT file at -c 32768 with --spec-type draft-mtp --spec-draft-n-max 1 (see the MTP section).

Honest caveat: at ±0.27 the tier-to-tier PPL gaps are inside the individual error bars. They are paired measurements on identical data so the ordering is meaningful, but do not read the quality differences as large — the whole spread is 1.8%.

Usage

llama-server -m Xing4.0-29B-A4B-Q4_0-ROCmFP4-COHERENT-noimatrix.gguf \
  -c 32768 -ngl 99 -fit off --jinja

--jinja is required for tool calling. -fit off with an explicit -c is the house rule on gfx1151 — auto-fit reads MemAvailable and can silently cut context or offload layers.

⚠️ This is a reasoning model — give it tokens

Thinking is on by default. A two-sentence answer consumed 569 completion tokens, and max_tokens: 200 returns empty content. Budget ≥ 600, or you will conclude the model is broken when it is only mid-thought. llama-server separates the trace into reasoning_content correctly.

Reasoning is often emitted in Chinese even when the answer is English — expected for this model, not a defect.

Tool calling

Verified working:

finish_reason: tool_calls
get_weather({"city":"Tokyo","unit":"celsius"})

Verification

Every number above is measured on the hardware named in the reproduction block. Nothing is estimated or carried over from a similar model.

Reference check. The port was validated against upstream transformers running the same model on the same tokens, not merely "looks coherent":

The port lands 0.7% below HF because llama.cpp computes the hyper-connection projection in F32 where HF casts to bf16 — the direction and magnitude were predicted in advance by an independent numpy reference before being measured.

⚠️ llama-perplexity scores only the second half of each window. A naive script scoring every token from position 1 gives 11.91 for this same model and data. If you compare against these numbers, match the scoring window or the comparison is meaningless.

Not tested

• —Contexts beyond 32K. The architecture declares 262144 but that context cannot be allocated on 128 GiB hardware (see MTP section); 32768 is what was served and measured.
• —Vulkan. All numbers are the ROCm backend.

MTP: measured, and it is the biggest lever on this model

Layer 40 is the model's MTP block and its weights are in these files (block_count = 41, nextn_predict_layers = 1). Measured 2026-09-19 on the STRIX_LEAN file, same flags, 589-token prompt generating 160 tokens, 32K context, quiet box, ROCm0:

Use `--spec-draft-n-max 1`. n-max 2 accepts slightly longer drafts (1.83 vs 1.65) and is still much slower, because the extra draft work costs more than it returns. Prefill pays 14.5% for the gain, so MTP favours generation-heavy traffic over long-prompt batch work.

⛔ MTP costs you half your context

The MTP draft context allocates its own full-size KV cache. Two consequences measured on a 128 GiB Strix Halo:

• —The declared 262144 context is not reachable on this hardware at all: the KV allocation asks for 209,920 MiB (205 GiB) and fails. That is ~0.80 MiB per token, uncompressed.
• —With MTP enabled, 131072 still fails (failed to create MTP context) and 32768 is what fits.

⚠️ `-ctk q8_0 -ctv q8_0` is silently ignored on this arch. At 131072 with both flags set the allocation was 104,960 MiB — exactly half the 262144 figure, i.e. pure context scaling with no quantisation benefit. Do not plan context around quantised KV here.

Reproduction

box     AMD Ryzen AI Max+ 395 "Strix Halo", gfx1151, ROCm 7.2.4, 123 GiB unified
build   github.com/kingjones30/ROCmFPX, arch xing4
source  XingChen-AGI/Xing4.0-29B-A4B (bf16 safetensors, 8307 tensors)
convert per-arch modular converter -> 936-tensor BF16 GGUF (62.4 GB)
quant   llama-quantize <src> <out> <ftype> 32
        COHERENT 102 | STRIX_LEAN 106 --output-tensor-type q6_K | FAST 103 --output-tensor-type q4_K

ppl     llama-perplexity -m <f> -f wiki.test.raw -c 512 --chunks 32 -ub 64 -ngl 99 -fit off
        (wikitext-2-raw test split)
speed   llama-bench -m <f> -p 512 -n 128 -ngl 99 -r 3      (quiet box, no co-resident seats)

⚠️ *`-ub` must be ≤ 128 when running an unquantized MoE on gfx1151 (BF16/F16 — e.g. when computing your own imatrix). The ROCm `mul_mat_id` sorted fallback returns wrong values there: `test-backend-ops -o MUL_MAT_ID` reports 1311/1320, with all 9 failures on f16/bf16. All 920 quantized MUL_MAT_ID cases pass, so the files in this repo are unaffected* — this only bites if you re-derive artifacts from the BF16 yourself.

Notes on the port

Getting this arch correct required fixing two defects in the hyper-connection ops that took wikitext PPL from 80.05 to 8.3608:

1. 1.Sinkhorn `eps` placement. eps was added to the value rather than the denominator. At 1e-6 that sounds harmless, but after exp(logits − rowmax) the legitimate entries in a row are ~2.5e-7 and ~6e-8, so it acts as a 5–17× floor that 20 Sinkhorn iterations amplify.
2. 2.comb orientation. hc_expand contracts ne1 and indexes the destination stream by ne0, so the doubly-stochastic mixing matrix arrived transposed — rows and columns both sum to ~1, so nothing crashed and every structural check passed while the streams mixed wrongly.

Credits

Base model © China Telecom Artificial Intelligence Technology Co., Ltd. (XingChen-AGI · GitHub), Apache-2.0. Quantized and arch-ported by kingjones777; these GGUFs inherit Apache-2.0.