kingjones777/Muse-Glimmer-30B-ROCmFPX-Q8_0-AGENT-GGUF

### 🔧 Runtime: build the ROCmFPX fork below Stock llama.cpp will not load this file. You need both the `muse-glimmer` architecture and the ROCmFP4 tensor types in one tree. Upstream `charlie12345/ROCmFPX` has the ROCmFP4 types but not muse-glimmer. Our fork has both: [`kingjones30/ROCmFPX`](https://github.com/kingjones30/ROCmFPX) — a fork of charlie12345/ROCmFPX, branch main. ``bash git clone https://github.com/kingjones30/ROCmFPX.git cd ROCmFPX cmake -B build -DGGML_HIP=ON -DGPU_TARGETS=gfx1151 -DGGML_NATIVE=ON -DCMAKE_BUILD_TYPE=Release cmake --build build --target llama-server llama-quantize -j$(nproc) ` Verified 2026-08-27 on gfx1151: clean clone → **0 build errors** → llama-server loads a muse-glimmer` ROCmFP4 GGUF from this family and generates coherent text.

Muse-Glimmer-30B — ROCmFPX 8-bit AGENT for AMD Strix Halo (gfx1151)

<!-- LEAD-CLAIM:START -->

✅ the complete `muse-glimmer` port — text graph, vision projector and chat parser — ships as a single applyable patch in this repo `muse-glimmer` is not an upstream llama.cpp architecture. Running it end-to-end takes three independent pieces of work; all three are in `patches/muse-glimmer-complete.patch` (20 files, 81,968 bytes, `git apply --check` clean).

<!-- LEAD-CLAIM:END -->

An 8-bit ROCmFPX quantisation of Muse-Glimmer-30B for AMD Ryzen AI Max+ 395 / Radeon 8060S / gfx1151, quantised from the BF16 GGUF — a lossless source, not a requantisation of a smaller file. ROCmFPX is a runtime tensor format that exists only in the ROCmFPX fork of llama.cpp.

Why this build?

• —Quantised from BF16, source revision verified by fetch (a4e59da52a7bc87ae7251dd5545c0dd437c44b68) rather than assumed
• —Output head and token embeddings pinned to q8_0 explicitly, not left to inherit the block type
• —The full architecture port is in the repo — you do not have to reconstruct it
• —Vision works through the BF16 projector built by stage 2 of that patch
• —Sizes are reported from stat, and checked against the --dry-run projection: the ~12.5 MiB delta is GGUF header, which is the signature of a complete file

Which file should I use?

If you want speed, take a 4-bit build — on this model the 4-bit files are roughly 3.5× the code-transform throughput of the 8-bits and less than half the size. The 8-bits are here for bit-count, not for speed.

⚠️ Decode on this model is workload-dominated, not variant-dominated. DFlash proposes long runs on repetitive and code-like text and very little on freeform prose, so a single tok/s figure is misleading — mean accepted length moves 2.5 → 7.1 across the same binary and the same weights. Quote a range for this model, not a point.

muse-glimmer-30B-Q8_0_ROCMFPX_AGENT.gguf and `muse-glimmer-30B-Q8_0_ROCMFPX.gguf` are within noise of each other (7.22 vs the other build's figure on the same harness). The AGENT routing lifts draft acceptance on MTP models; this model uses DFlash, so there is nothing for it to win here. Choose on size.

Quick start

hf download kingjones777/Muse-Glimmer-30B-ROCmFPX-Q8_0-AGENT-GGUF --local-dir glimmer

The DFlash drafter is not duplicated here — pull it from the 4-bit repo, which carries every drafter and projector variant:

hf download kingjones777/Muse-Glimmer-30B-ROCmFP4-Strix-Halo-DFlash-GGUF \
  --include "dflash-ROCmFP4-STRIX.gguf" --local-dir glimmer

⚠️ hf download silently ignores --include when given more than one pattern — one call per file.

llama-server \
  -m glimmer/muse-glimmer-30B-Q8_0_ROCMFPX_AGENT.gguf \
  --mmproj glimmer/mmproj-muse-glimmer-30B-BF16.gguf \
  --spec-type draft-dflash --model-draft glimmer/dflash-ROCmFP4-STRIX.gguf \
  --spec-draft-ngl 99 \
  -ngl 999 -c 4096 -fa off -fit off --jinja \
  --host 127.0.0.1 --port 8080

Three flags that matter more than which file you pick

⚠️ This head is DFlash, not MTP. Read mean accepted length, and do not pass MTP flags to it.

Vision

Verified on spatial placement rather than plausible-sounding output: a solid-colour image is scored on whether the model names the colour that is actually there.

Serve with `-fa off` and the BF16 projector from this repo (mmproj-muse-glimmer-30B-BF16.gguf, 3,849,174,048 bytes, 809 tensors).

⚠️ Minimum useful image size is 28×28 px. The preprocessor snaps to patch 14 × merge 2, so anything smaller collapses to a single merge token, carries no spatial signal, and the model reports the dominant colour of the padded canvas. Feed 256×256 or larger. llama-mtmd-cli behaves identically — this is the preprocessing geometry, not the projector.

Files

🩹 The muse-glimmer architecture port — complete, three stages

patches/muse-glimmer-complete.patch — 81,968 bytes, 20 files, git apply --check clean against `charlie12345/ROCmFPX`.

git clone https://github.com/charlie12345/ROCmFPX.git && cd ROCmFPX
git apply --check ../patches/muse-glimmer-complete.patch && \
git apply           ../patches/muse-glimmer-complete.patch
cmake -S . -B build-muse -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151 \
      -DLLAMA_BUILD_WEBUI=OFF -DCMAKE_BUILD_TYPE=Release
cmake --build build-muse --target llama-server --target llama-quantize \
      --target llama-mtmd-cli -j 6

muse-glimmer is not an upstream architecture. Running it end-to-end takes three independent pieces of work, and all three are in this patch.

Stage 1 — text graph and conversion

⚠️ convert_hf_to_gguf.py does not know MuseGlimmer. The converter is conversion/muse_glimmer.py, driven through conversion.get_model_class.

Stage 2 — vision projector (clip / mtmd)

This stage is what makes --mmproj work. Use the BF16 projector in this repo (mmproj-muse-glimmer-30B-BF16.gguf, 3,849,174,048 bytes, 809 tensors, clip.projector_type = muse-glimmer, merge 2, patch 14, image_size 896).

Stage 3 — chat parser

common/chat.cpp — common_chat_params_init_muse_glimmer, PEG_NATIVE.

Without it the model's to=self<|message|> control sequence is emitted into content. With it, content is clean: "The capital of Japan is Tokyo."

Build note

cmake's source GLOB is configure-time. After the patch adds src/models/muse-glimmer.cpp you must re-run the cmake -S . -B build-muse configure step, not just --build. -DLLAMA_BUILD_WEBUI=OFF avoids a node/npm requirement.

Applying to a different base commit

The patch header names commit 3edc3d3, and it applies cleanly to later revisions (verified on b41ce12). On trees where cohere2moe and bailing_hybrid model sources are absent, their factory cases in llama-model.cpp reference symbols that do not exist in that tree — build those two out, or apply on 3edc3d3 where their .cpp files are present. The muse-glimmer factory case and graph are independent of both.

Quantization methodology

# 1. convert BF16 safetensors -> GGUF (the muse-glimmer converter is not in
#    convert_hf_to_gguf.py; it ships as conversion/muse_glimmer.py in the patch)
python -c "from conversion import get_model_class; ..."   # driver, see patch README

# 2. measure the real BPW before committing
llama-quantize --dry-run muse-glimmer-30B-BF16.gguf /tmp/x.gguf Q8_0_ROCMFPX 8

# 3. quantize with the patched ROCmFPX build (only it has ggml types 100-106)
llama-quantize --output-tensor-type q8_0 --token-embedding-type q8_0 \
  muse-glimmer-30B-BF16.gguf muse-glimmer-30B-Q8_0_ROCMFPX.gguf Q8_0_ROCMFPX 16

Source: meta-models/Muse-Glimmer-30B, revision `a4e59da52a7bc87ae7251dd5545c0dd437c44b68` (fetched and verified, not assumed) → muse-glimmer-30B-BF16.gguf 55,725,514,112 bytes. Quantized from BF16 only — this is not a requantization of a smaller file.

--output-tensor-type and --token-embedding-type are both set explicitly so the output head and token embeddings land at q8_0 rather than inheriting the block type.

Not yet measured

Listed so nobody mistakes absence for a pass:

Known issues

1. 1.Stock llama.cpp cannot load these files. muse-glimmer is not an upstream architecture and ROCmFP4/ROCmFPX are not upstream tensor types. Both come from the patch in this repo.
2. 2.`-fa off` is required for the vision path on gfx1151. Text-only serving runs fine with -fa on.
3. 3.Images below 28×28 px collapse to a single merge token. The preprocessor snaps to patch 14 × merge 2, so an 8×8 input carries no usable spatial signal and the model reports the dominant colour of the padded canvas. Feed images at 256×256 or larger.
4. 4.Vulkan / CUDA / CPU cannot load these files — ROCmFP4/ROCmFPX are ROCm-only formats.

Independent results

None yet. If you run this build, please open a discussion with hardware, GPU, ROCm version, runtime commit, exact command, context, prompt-processing tok/s, generation tok/s and peak RAM. Independent reproductions will be listed separately from author benchmarks and carry more weight.

License and attribution

Base model and its licence are the Muse team's. ROCmFP4 / ROCmFPX quantisation types are from the ROCmFPX fork of llama.cpp. This repository contains the quantised weights, the architecture port and the measurements above.

<!-- CREDITS:START -->

Acknowledgements

[ROCmFPX](https://github.com/charlie12345/ROCmFPX) — maintained by [`charlie12345`](https://github.com/charlie12345) The ROCmFP4 / ROCmFPX tensor formats (ggml types 100–106) exist only in this fork. Every file here was produced with its llama-quantize and runs on its runtime. Licensed MIT, based on upstream llama.cpp.

[llama.cpp](https://github.com/ggml-org/llama.cpp) — ggml-org and contributors The inference engine, GGUF format and conversion tooling everything here is built on.

[AMD ROCm](https://github.com/ROCm/ROCm) — the compute platform these builds target, on gfx1151 / Radeon 8060S.

[Muse](https://huggingface.co/meta-models) — the base model and its licence are theirs. This repository contributes the architecture port, quantisation and measurement only.

If you use these files, please credit ROCmFPX alongside this repository.

<!-- CREDITS:END -->