kingjones777/Laguna-S-2.1-ROCmFP4-STRIX_LEAN-GGUF

Laguna-S-2.1 ROCmFP4 STRIX_LEAN (GGUF) — AMD Ryzen AI Max+ 395 / Strix Halo / gfx1151

First public-ready Laguna-S-2.1 ROCmFP4 "Strix Lean" quant for AMD Ryzen AI Max+ 395 (gfx1151 / Radeon 8060S).

⚠️ Not compatible with upstream llama.cpp. Requires the charlie12345/ROCmFPX fork built with HIP + ROCmFP4 kernels.

Files

Base model: poolside/Laguna-S-2.1 F16 source: poolside/Laguna-S-2.1-GGUF laguna-s-2.1-F16.gguf (235202258240 bytes)

Hardware / stack (validated)

• —Box: Ryzen AI Max+ 395, gfx1151, 128 GB unified
• —ROCm 7.2.4 (/opt/rocm-7.2.4, clang 22)
• —Fork: charlie12345/ROCmFPX @ 3edc3d31ee5ebcea47fd7e0f42c89767bb4245db

Build recipe (HIP-only)

export PATH=/opt/rocm-7.2.4/bin:$PATH
export HIP_PATH=$(hipconfig -R)
export HIPCXX=$(hipconfig -l)/clang
cmake -B build \
  -DGGML_HIP=ON -DGPU_TARGETS=gfx1151 \
  -DGGML_HIP_ROCWMMA_FATTN=ON -DGGML_HIP_NO_VMM=ON -DGGML_HIP_MMQ_MFMA=ON \
  -DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=OFF -DLLAMA_BUILD_WEBUI=OFF
cmake --build build -j 4   # prefer -j 4..8 on 128GB Strix; avoid -j32

Quantize

build/bin/llama-quantize \
  laguna-s-2.1-F16.gguf \
  Laguna-S-2.1-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  Q4_0_ROCMFP4_STRIX_LEAN 8
# quant size = 59739.82 MiB (4.26 BPW)

Runtime (required)

export LD_LIBRARY_PATH=<build/bin>:/opt/rocm/lib
export HSA_OVERRIDE_GFX_VERSION=11.5.1
export GGML_HIP_ENABLE_UNIFIED_MEMORY=1

llama-server --host 127.0.0.1 --port 8099 \
  --n-gpu-layers 999 --flash-attn on -dio --no-warmup --jinja \
  --model Laguna-S-2.1-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  --ctx-size 65536 --cache-type-k q8_0 --cache-type-v q8_0 --parallel 1 \
  --temp 0.2 --top-p 0.95 --top-k 20 --min-p 0.05 --repeat-penalty 1.1 --repeat-last-n 256 \
  --chat-template-kwargs '{"enable_thinking":true}' --reasoning-budget 512

`-dio` is required for reliable cold load of large Laguna GGUFs on this stack (mmap path can hang).

A/B vs Q4KM (same binary, same flags, cold load)

• —Decode speedup: +62.6% @ ~8K, +43.6% @ ~32K
• —Size: −18% (58.3 vs 71 GiB)
• —Prompt processing slightly slower on ROCmFP4 (−14% / −12%)

Quality / tools / reasoning

Shared prompts (reasoning, code, tool-call plan, short math with thinking):

Did we lose anything? No quality regression observed on the four shared prompts. We gain decode speed and lose model file size; prefill is slightly slower.

⚡ DFlash speculative decoding — +23% decode (added 2026-08-08)

DFlash works with this quant and is worth turning on: 32.6 → 40.1 tok/s (1.234×) on my box, same binary, same flags, only the speculation changed.

Draft head: [wimmmm/poolside-Laguna-S-2.1-DFlash-GGUF](https://huggingface.co/wimmmm/poolside-Laguna-S-2.1-DFlash-GGUF), Q8_0, 1.19 GB (the GGUF twin of poolside/Laguna-S-2.1-DFlash-NVFP4). Keep the head at Q8_0 — draft quality drives acceptance.

Two things will stop you cold. Both are fixable.

1. The fork commit pinned above is too old for DFlash

Building at 3edc3d3 (the commit in the recipe above) gives:

error loading model: done_getting_tensors: wrong number of tensors; expected 76, got 69

The 7 unmapped tensors are 6× blk.N.attn_gate.weight plus enc.aux_norm.weight. Current ROCmFPX main knows them; that commit does not. Quick check on your own build:

grep -ac aux_norm    build/bin/libllama.so   # newer: non-zero, old: 0
grep -ac decoder_arch build/bin/libllama.so  # newer: non-zero, old: 0

Build ROCmFPX main (add -DLLAMA_BUILD_WEBUI=OFF — the WebUI asset step fails without Node/npm and takes llama-server down with it at 100%). You can keep your existing build for everything else and point only Laguna at the new one.

2. The DFlash head declares a target layer that doesn't exist

The head ships dflash.target_layers = [2, 11, 20, 30, 39, 48], but Laguna-S-2.1 is block_count = 48, so valid indices are 0–47. llama.cpp asserts and aborts:

llama-context.cpp: GGML_ASSERT(lid < model.hparams.n_layer) failed

vLLM tolerates 48 as "the final hidden state"; llama.cpp does not. Patch the last element to 47 — it is a single 4-byte value of the same width, so it rewrites in place with no re-encode:

import struct, shutil
src="laguna-s-2.1-dflash-Q8_0.gguf"; dst="laguna-s-2.1-dflash-Q8_0-fix47.gguf"
shutil.copy(src, dst)
f=open(dst,"r+b"); assert f.read(4)==b"GGUF"
struct.unpack("<I",f.read(4)); struct.unpack("<Q",f.read(8))
n_kv=struct.unpack("<Q",f.read(8))[0]
def rs():
    n=struct.unpack("<Q",f.read(8))[0]; return f.read(n).decode()
SZ={0:1,1:1,7:1,2:2,3:2,4:4,5:4,6:4,10:8,11:8,12:8}
for _ in range(n_kv):
    k=rs(); t=struct.unpack("<I",f.read(4))[0]
    if t==9:
        et=struct.unpack("<I",f.read(4))[0]; ln=struct.unpack("<Q",f.read(8))[0]
        start=f.tell()
        if k=="dflash.target_layers":
            fmt={4:"<I",5:"<i",10:"<Q",11:"<q"}[et]
            f.seek(start+(ln-1)*SZ[et]); f.write(struct.pack(fmt,47))
            print("patched last target layer -> 47"); break
        if et==8:
            for _ in range(ln): rs()
        else: f.seek(SZ[et]*ln,1)
    elif t==8: rs()
    else: f.seek(SZ[t],1)
f.close()

Serving with DFlash

llama-server --host 127.0.0.1 --port 8099 \
  --n-gpu-layers 999 --flash-attn on -dio --no-warmup --jinja \
  --model Laguna-S-2.1-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  --spec-type draft-dflash \
  -md laguna-s-2.1-dflash-Q8_0-fix47.gguf -ngld 999 \
  --spec-draft-n-max 4 --spec-draft-p-min 0.5 \
  --ctx-size 65536 --cache-type-k q8_0 --cache-type-v q8_0 --parallel 1

⚠️ You must set `--spec-draft-n-max`. The default of 16 exceeds the head's trained block size of 15 and the server aborts in the DFlash constructor before it ever listens.

Credit for the GGUF draft head to wimmmm; the target-layer patch and the build-version finding are mine.

License

Follow the base model (poolside/Laguna-S-2.1) license terms.

<!-- PEER-TABLE:START -->

Other public builds of this model

Compiled from Hugging Face repository metadata — file sizes, shipped files, quant variant as named by each repo. No third-party build was run or benchmarked here, so this table makes no speed or quality claim about any of them. It is here so you can see the size and format options at a glance and pick what fits your hardware.

Base model: [`poolside/Laguna-S-2.1`](https://huggingface.co/poolside/Laguna-S-2.1). Generated from Hub metadata; download counts move over time.

<!-- PEER-TABLE:END -->

<!-- CREDITS:START -->

Acknowledgements

This build would not exist without the work below. Please star and follow these projects — the quantisation format used here is their engineering, not mine.

[ROCmFPX](https://github.com/charlie12345/ROCmFPX) — maintained by [`charlie12345`](https://github.com/charlie12345) / `caf` The ROCmFP4 / ROCmFPX tensor formats (ggml types 100–106) exist only in this fork. Every ROCmFP4 file in this repository was produced with its llama-quantize, and runs on its runtime. The fork also credits collaborators ciru-ai, Tom Turney, PlunderStruck and Aydan S., and acknowledges AMD for hardware support. Licensed MIT, based on upstream llama.cpp.

[llama.cpp](https://github.com/ggml-org/llama.cpp) — ggml-org and contributors The inference engine, GGUF format and conversion tooling everything here is built on.

[AMD ROCm](https://github.com/ROCm/ROCm) The compute platform these builds target — ROCm 7.2.4 on gfx1151 / Radeon 8060S.

Base model authors — see base_model in the metadata above; all model weights, licences and capabilities are theirs. This repository contributes quantisation and measurement only.

If you use these files, please credit ROCmFPX alongside this repository.

<!-- CREDITS:END -->