jcbtc/Laguna-S-2.1-Chadrock-ROCmFP4-StrixKVSpine-V4-GGUF

<p align="center"> <img src="assets/laguna.png" alt="Laguna S 2.1 118B Chadrock ROCmFP4 for AMD Strix Halo" width="100%"> </p>

Laguna S 2.1 118B Chadrock ROCmFP4 StrixKVSpine V4 — Runtime V3

An AMD-optimized, quality-protected ROCmFP4 quant of Poolside Laguna S 2.1, built for local agentic coding on Ryzen AI Max+ 395 / Radeon 8060S Strix Halo.

This V4 recipe fits a 118B-total-parameter, approximately 8B-active model into a 60.945 GiB GGUF at 4.453 effective BPW, while retaining a tested 131,072-token safe serving profile. It is 12.95% smaller than Poolside's official Q4KM GGUF and reached 35.62 tokens/second during the complete 128K V2 stability gate retained by Runtime V3.

[!IMPORTANT] This GGUF uses ROCmFP4 tensor types and Laguna architecture support. It is built for the Laguna-enabled Ciru ROCmFPX Runtime V3 at commit 54f5fe06c74350fb8b6aec21d8749071bc195bdb. It does not load in stock upstream llama.cpp. Runtime V3 fixes Laguna tool-call grammar/parser agreement and removes a non-portable chat-template extension while retaining the V2 Vulkan stability safeguards and safe serving defaults. The V4 GGUF weights are unchanged; existing users do not need to download the 60.945 GiB model again.

At a glance

Runtime V3 patch notes

Runtime V3 adds parser and grammar fixes on top of the V2 Vulkan runtime:

• —ports llama.cpp PR #24835 so generated JSON values no longer carry trailing grammar whitespace that the final PEG tool parser rejects;
• —aligns PEG JSON-array comma whitespace handling with the generated grammar;
• —adds a Laguna/Pi regression for an edit call containing an array<object> argument and source-code strings;
• —keeps server checkpoints host-backed, preventing the on-device checkpoint destruction fault seen when a long hybrid/SWA context is invalidated;
• —explicitly disables context checkpoints in the validated Laguna launcher while leaving the normal KV and prompt caches available;
• —retains the V2 RADV DeviceLost safeguards and validated 128K defaults.

Focused V3 validation on Ryzen AI Max+ 395 / Radeon 8060S with Mesa RADV 26.1.2 passed JSON-schema grammar conversion, automatic parser selection, the nested Pi edit-call regression, and the Laguna architecture test. Real-model smokes returned Paris., preserved the nested edit tool call, completed an 8,061-token functional smoke with the expected omega response, and released the slot after cancellation at 82% of a 15,000-token request. A follow-up request returned Paris. with no VK_ERROR_DEVICE_LOST.

The checkpoint repair was exercised separately with checkpoints explicitly re-enabled and prompt-cache RAM left at 8192 MiB. A 120,045-token request created 16 hybrid/SWA checkpoints; an unrelated follow-up forced pos_next=0 and erased all 16, then returned OK.. The server remained healthy and shut down cleanly without a core dump. Checkpoints nevertheless remain disabled in the public profile until this gate is repeated across multi-turn workloads.

V2 Vulkan stability baseline

The first runtime release could lose the Vulkan device during a very deep Flash Attention prefill on RADV/Strix Halo. Lowering the graph-node submission ceiling was not enough: matched 100-node and 10-node controls both reached an AMD compute-ring timeout after approximately 77–78 minutes.

V2 fixes the operator-level problem by splitting a large Flash Attention X grid into shorter Vulkan dispatch commands while preserving global workgroup IDs and output offsets.

V2 validation on Ryzen AI Max+ 395 / Radeon 8060S with Mesa RADV 26.1.2:

The 8K V2 row improved prompt processing by 10.57% over the matched unsplit 10-node control (318.70 tok/s), with effectively unchanged generation speed. Deterministic split and unsplit test generations were byte-identical after removing their timing lines.

Runtime V2 also adds:

• —GGML_VK_FA_MAX_WORKGROUPS_X_PER_DISPATCH;
• —GGML_VK_MAX_NODES_PER_SUBMIT;
• —first-failure graph node/operator context;
• —no new Vulkan submissions or failed fence waits after DeviceLost;
• —portable crash collection and a supervised launcher;
• —an explicit warning when selecting the experimental 256K lane.

Why this release

The goal was not simply to make Laguna smaller. StrixKVSpine V4 protects the tensors that were most sensitive in our Laguna experiments while using the fast ROCmFP4 path where it delivered the best memory and throughput return:

• —attention K/V, attention gates, dense block 0, shared experts, and a nine-layer expert-down spine remain protected;
• —attention Q/O and non-spine packed experts use the fast ROCmFP4 path;
• —the output tensor remains Q6_K;
• —F16/F16 KV cache is retained for the validated 128K profile.

The resulting model is 9.066 GiB smaller than the official Poolside Q4_K_M while matching or improving that baseline on most of the retained quality checks.

Results against Poolside Q4KM

These are direct local comparisons against Poolside's official laguna-s-2.1-Q4_K_M.gguf, using the same benchmark tasks. Scores are reported individually rather than blended into a synthetic aggregate.

The hero's rounded quality figure is the matched Tool-Eval result: 80 accepted calls versus 62, a 29.0% increase.

BigCodeBench follow-up

The official V4 BigCodeBench run used strict greedy decoding and scored 37/148, with seven length-capped repetition loops. Under the release sampler, all seven completed naturally and two additional tasks passed. The resulting sampler-corrected diagnostic is 39/148, tied with Q4KM. The table retains the official 37/148 score.

On the same 148 BigCodeBench prompts, V4 measured:

The table reports per-token throughput. The greedy run generated more than twice as many completion tokens because of the seven loops, so end-to-end wall time from that run is not used as the speed headline.

Recommended serving profile

Linux support

The runtime builds natively on Linux x86-64. NixOS is the currently validated production build environment; Ubuntu 24.04 LTS and Debian 12+ are the primary documented user path. The repository also provides native dependency paths for Fedora/Rocky/AlmaLinux and Arch/Manjaro.

Clone and pin Runtime V3 exactly:

git clone --branch agent/laguna-s21-runtime-v3 --depth 1 \
  https://github.com/ciru-ai/ROCmFPX.git
cd ROCmFPX
git checkout --detach 54f5fe06c74350fb8b6aec21d8749071bc195bdb
test "$(git rev-parse HEAD)" = \
  "54f5fe06c74350fb8b6aec21d8749071bc195bdb"

The distro-aware helper prints the native package command before making any change:

scripts/install-laguna-vulkan-deps.sh
scripts/install-laguna-vulkan-deps.sh --install

Ubuntu and Debian users can install directly:

sudo apt-get update
sudo apt-get install -y \
  git cmake ninja-build build-essential glslc \
  libvulkan-dev vulkan-tools spirv-headers mesa-vulkan-drivers

<details> <summary>Fedora, Rocky Linux, and AlmaLinux</summary>

sudo dnf install -y \
  git cmake ninja-build gcc gcc-c++ glslc \
  vulkan-loader-devel vulkan-headers spirv-headers \
  vulkan-tools mesa-vulkan-drivers

</details>

<details> <summary>Arch, Manjaro, and EndeavourOS</summary>

sudo pacman -S --needed \
  git cmake ninja base-devel shaderc \
  vulkan-icd-loader vulkan-headers spirv-headers \
  vulkan-tools vulkan-radeon

</details>

<details> <summary>NixOS</summary>

nix --extra-experimental-features 'nix-command flakes' profile add \
  nixpkgs#git nixpkgs#cmake nixpkgs#ninja nixpkgs#gcc \
  nixpkgs#shaderc nixpkgs#vulkan-headers nixpkgs#vulkan-loader \
  nixpkgs#spirv-headers

</details>

Then verify Vulkan and build the pinned Release runtime:

vulkaninfo --summary
JOBS=8 BUILD_TYPE=Release scripts/build-laguna-strix-vulkan.sh

This produces a static Vulkan build with llama-server, llama-cli, llama-bench, and llama-quantize.

Run the release checks from the repository root:

build-laguna-strix-vulkan/bin/test-json-schema-to-grammar
build-laguna-strix-vulkan/bin/test-chat-auto-parser
build-laguna-strix-vulkan/bin/test-chat \
  --template poolside-Laguna-S-2.1.jinja
build-laguna-strix-vulkan/bin/test-llama-archs

The complete Linux and V3 guide contains the Fedora, Arch, and NixOS commands.

Start Laguna with the validated 128K V3 profile

The supervised launcher is recommended on RADV. It runs the driver preflight, uses the safe V3 settings, and preserves DeviceLost evidence:

scripts/run-laguna-vulkan-supervised.sh \
  /path/to/laguna-s-2.1-ROCmFP4-StrixKVSpine-v4.gguf

The launcher applies the measured single-slot Strix Halo configuration: Vulkan0, full offload, row split, Flash Attention, 131,072 context, F16/F16 KV, batch 2048, ubatch 512, node cap 10, FA dispatch width 4, 16 threads, thinking off, context checkpoints disabled, and this sampler:

{
  "temperature": 1.0,
  "top_p": 1.0,
  "top_k": 20,
  "min_p": 0.0,
  "repeat_penalty": 1.0,
  "seed": 42
}

The server listens on 127.0.0.1:8080. Check it with:

curl http://127.0.0.1:8080/health
curl http://127.0.0.1:8080/v1/models

The direct runner uses the same V3 safe defaults without supervision:

scripts/run-laguna-s21-rocmfp4-v4.sh /path/to/model.gguf

Context checkpoints are separate from the normal KV cache. The launcher now passes --ctx-checkpoints 0 because the hybrid/SWA checkpoint path has not yet completed the full repeated 128K multi-turn and cache-replay qualification. The runtime also moves checkpoint payloads back to host memory, matching the safer upstream design. Advanced diagnostic runs can opt in with CTX_CHECKPOINTS=N; this is not part of the validated public profile yet.

The model's 256K capacity remains available only as an explicit experimental lane:

STABILITY_MODE=performance \
  scripts/run-laguna-s21-rocmfp4-v4.sh /path/to/model.gguf

That command prints a warning because 256K has not yet passed the full-depth prefill, multi-turn, and cache-replay gates.

The complete production recipe is preserved in the ROCmFPX Laguna Runtime V3 guide.

Example request

After starting the compatible server:

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "laguna-s21-rocmfp4-strixkvspine-v4",
    "messages": [
      {
        "role": "user",
        "content": "Refactor this Python API client to add bounded retries, typed errors, and tests."
      }
    ],
    "temperature": 1.0,
    "top_p": 1.0,
    "top_k": 20,
    "min_p": 0.0,
    "seed": 42
  }'

Artifact integrity

Credits

Charlie / charlie12345 / caf

Enormous thanks to Charlie (`charlie12345`) for the amazing ROCmFP4 codebook and the experimental ROCmFPX work that made this release possible. We could not have built this release without him.

Please support and credit his work when building on ROCmFP4 or ROCmFPX.

Poolside

Thank you to Poolside for creating and releasing the remarkable Laguna S 2.1 model and its official GGUF collection. Laguna is the foundation of everything here; this release is a quantized, hardware-targeted derivative, not a new base model.

Ciru / Chadrock

Ciru developed the Laguna-specific StrixKVSpine tensor-protection recipe, performed the calibration and quantization, built the Strix Halo Runtime V2 stability baseline and Runtime V3 parser/template layer, and ran the retained quality, performance, deep-context, tool-call, and cancellation validation.

License and use

This derivative follows the base model's OpenMDW 1.1 license and Poolside's published model terms. Review the base model card, license, and acceptable-use requirements before deployment.

Benchmark results describe this exact file, runtime, hardware, and sampler configuration. Performance and memory behavior will vary across drivers, backends, hardware, context lengths, and workload shapes.