Geometric-AI/Muse-Glimmer-30B-geoquant-GGUF

Muse-Glimmer-30B — geo-quant measured GGUF builds

Two GGUF re-quantizations of `meta-models/Muse-Glimmer-30B` (dense 29.8B agentic VLM, LM tower only), built by allocating bytes from measured per-tensor damage rather than from a named preset.

• —`muse-v4` sits 812 KB under the official 17 GB build and scores above it on every repeat run. Stock K-quants only — no custom loader.
• —`muse-lowbpw-r1` matches the official build's measured band using 69 % of its bytes, by letting two custom low-bit formats compete against stock K-quants per tensor. Needs one of the two runtimes below — mainline llama.cpp rejects the file rather than misreading it.

Both are text-only on their own. For image input, pair with the vendor's mmproj-kquant.gguf from the official GGUF repo.

What we compared against

Every number below is our measurement of their published artifact through the same harness, not a vendor-reported score. We did not re-quantize anyone else's model or tune against their choices.

The first-party build is the one the vendor ships and the one our byte ceiling is sourced from. Unsloth is included as an independent, widely-used community quantizer so the comparison is not purely against the model's own authors.

UD-IQ2_XS is within 0.11 % of muse-lowbpw-r1's size (theirs is 13 MB larger), which makes that pair the cleanest head-to-head in the set. UD-Q4_K_XL is the nearest Unsloth build to muse-v4 but is 5.2 % smaller, so it is not a byte-matched comparison and is not reported as one.

Benchmark

A 122-item agentic golden suite — MATH 40 / CODE 40 / TOOL 30 / AGENT 12 — scored by a deterministic grader. Corpus perplexity and token agreement are not used as gates; they are smoke tests that do not move with agentic ability.

Every artifact is served identically and scored by the same grader:

• —llama.cpp llama-server, master `dd1ea52`, CUDA arch 90
• —-ngl 99 --ctx-size 16384 --parallel 4 --special
• —greedy sampling; chat template rendered client-side from the HF tokenizer
• —per-axis max_new_tokens recorded in each results file

--special is load-bearing: serving with control tokens stripped measures TOOL 0/30, because ATEM tool calls are delimited by them.

Controls

The gate is only worth reading because it has been shown to move in both directions:

• —Positive control — official kquant-17gb scores 101/122.
• —Negative control — a uniform Q2_K crush (--pure, 2.63 bpw) scores 1/122. The gate fails hard on a broken artifact.
• —Instrument validation — grader --self-check 122/122 against canonical answers, plus bf16 anchor reproduction re-run after every grader change.

Reported ranges are repeat runs of the same artifact at the same config (n given), not confidence intervals. Single-run numbers are labelled n=1.

Results

Claims, each stated at the strength the data supports:

• —muse-v4 beats the official build at byte parity. Worst run (103) ≥ best official run (101), and it is 812 KB smaller. TOOL exceeds even the bf16 reference (27 vs 26) in all three runs.
• —muse-lowbpw-r1 beats the size-matched Unsloth build. 101–103 against 96–98 at 13 MB less. The bands do not overlap: our worst run clears their best by 3 items. The gap is mostly MATH (32·30·31 vs 28·29·29).
• —muse-lowbpw-r1 matches the official band on 69 % of the bytes. Worst run (101) equals the official build's best. A match claim, not a beat claim.
• —muse-lowbpw-r1 also matches UD-Q4_K_XL on 72 % of its bytes (101–103 vs 100–102). The bands overlap, so this is a match, not a win — but the byte asymmetry runs in our favour, so it is not confounded.
• —AGENT is the standout. muse-lowbpw-r1 holds bf16 parity (8·8·8) and is the only quantized build here that does — official 6·7, UD-Q4KXL 6·6·6, UD-IQ2_XS 7·7·7, muse-v4 6·7·7. That is the axis that normally degrades first.

Stated fairly in the other direction: Unsloth's UD-Q4_K_XL matches the first-party 17 GB build (100–102 vs 100–101) while being 5.2 % smaller — a good result for it. muse-v4 scores above it, but muse-v4 is 878 MB (5.5 %) larger, so that particular pairing is not byte-matched and we do not claim it as a win.

Serving throughput

Measured with bench_muse_decode in geo-lucebox (prefill once, time N single-token steps, no sampling), batch 1:

muse-lowbpw-r1 decode on CUDA was 26.0 tok/s until a kernel fix landed upstream of this release (2.06×, bit-identical output). On AMD the same change is neutral, so those figures reflect the current kernels on both.

With speculative decode against the vendor DFlash drafter (dflash_server --draft, full 122-item suite on H200), muse-lowbpw-r1 reaches 81.0 tok/s — 1.54× its own autoregressive 52.6 — at 103/122, inside the ±2 the suite is noisy to. The gain is workload-dependent: it tracks draft acceptance, which is higher on reasoning and code traffic (0.32 on the suite) than on open-ended chat (0.15–0.24), so short conversational turns will see less than 1.54×.

On the geo-lucebox runtime, batched multiplies for qtypes 105/106 are routed per multiply on batch width: narrow batches (speculative verify) take the quantized MMQ kernels, wide ones (prefill) take dequantize + dense GEMM, because neither wins both regimes. It is automatic — there is no flag to set — and it is why the speculative figure above does not come at the cost of prefill. Details in `server/docs/MUSE_GLIMMER.md`.

The llama.cpp fork below loads and runs the same artifact, but does not carry that routing — its 105/106 multiplies take the dequantize path at every batch width. The throughput figures in this section were measured on geo-lucebox.

Running them

`muse-v4` — mainline llama.cpp, nothing special:

llama-server -m muse-v4.gguf -ngl 99 --ctx-size 16384 --special

`muse-lowbpw-r1` — requires a runtime that understands qtypes 105/106 (Q3_1_ROCMFP3_MIX / Q2_1_ROCMFP2_MIX) and the geoquant.dmix2.sidecar KV. Mainline llama.cpp will reject the file rather than misread it. Two public options:

1. 1.llama.cpp fork — `GeometricAGI/llama.cpp`, branch `muse-rocmfpx-cuda` (CUDA + ROCm):

llama-server -m muse-lowbpw-r1.gguf -ngl 99 --ctx-size 16384 --special

1. 1.[`GeometricAGI/geo-lucebox`](https://github.com/GeometricAGI/geo-lucebox) dflash_server — a standalone CUDA/HIP server with native support for this model family (ATEM chat format, tool-call parsing, DFlash speculative decode against the vendor drafter). See server/docs/MUSE_GLIMMER.md there.

• —Full GPU offload of every mix layer is mandatory. Decode for these qtypes is GPU-only; the loader refuses partial offload rather than silently producing wrong numbers. On unified-memory parts (Strix Halo) a host pointer can read something valid-but-wrong instead of faulting, which is why the refusal is by name and not by fault.
• —The sidecar KV must exactly cover the resident mix tensors; any mismatch refuses the file.

Chat template

ATEM: <|start|>role[ to=recipient]<|message|>…<|eom|>/<|eot|>, with <atem:function_calls> blocks and a to=self reasoning channel. Serve with --special. <|eom|> ends a segment, not the turn — treating it as a stop token truncates every reply at the end of its reasoning.

Method, briefly

• —Allocation is measured, not hand-picked. Every byte decision comes from a per-tensor damage table (relative layer-output error on 128×2048-token activations of a domain calibration bundle) fed to a knapsack under a sourced byte ceiling. For muse-lowbpw-r1 the custom low-bit rungs compete in that same knapsack and are selected only where measurement favours them — which is why they appear on 71 of 418 tensors rather than everywhere.
• —Calibration is domain-representative and contamination-audited. 400 agentic traces (math reasoning, code, ATEM tool episodes, chat), audited against the golden suite by 60-char shingle containment: zero hits.
• —No external imatrix files are consumed. Rungs ≥4 bpw are encoded stock-unweighted; sub-4 bpw rungs are weighted by per-channel calibration moments captured in-process (worth +3 gate items at identical bytes here).

Limitations

• —LM tower only. Vision needs the vendor mmproj; we did not quantize or evaluate the perception encoder.
• —Single model family, single scale. These results are a strong prior for this model, not a general law about the formats.
• —The 122-item suite is small enough that ±2 items is noise; that is why bands from repeat runs are reported instead of single numbers, and why the claims above are phrased against worst/best runs rather than means.
• —muse-lowbpw-r1 needs a non-mainline loader (two public options above). If that is a problem, use muse-v4.

Reproducing

The serving side is public: the llama.cpp fork branch carries the qtype-105/106 decode kernels and sidecar registration, and geo-lucebox carries the native server plus its test suite. The quantization harness that produced these files (measured per-tensor damage → knapsack allocation under a sourced byte ceiling, golden-suite gating) is not public; the Method section above states what it does, and the numbers in this card are reproducible against the published artifacts with any harness that scores at the stated serving geometry.

License

Apache 2.0, inherited from the base model — a quantization is a derivative and cannot be relicensed. Sourced from `meta-models/Muse-Glimmer-30B`, whose card, LICENSE file, and model-card body all state Apache 2.0. Users remain subject to the base model's usage policy, which travels with the weights and is not altered by re-quantization.