AtomicChat/Muse-Glimmer-30B-GGUF

Muse Glimmer 30B — GGUF

GGUF builds of Meta's Muse Glimmer 30B, made from the original BF16 weights with a calibrated importance matrix and a bit layout chosen by measurement.

[!NOTE] Support was merged into llama.cpp master in #26841 (commit 62bf73d). You need a build from master — the last tagged release predates it.

Files

[!TIP] AD-Q6_K at 28.3 GB is dominated by plain Q8_0 at 29.6 GB, which is 1.3 GB larger and 31 % more accurate. Take Q8_0 instead unless you need exactly that size.

imatrix/ holds the importance matrix, layouts/ the per-tensor type map of every file, experiments/ the control builds. All logs: `AtomicChat/Muse-Glimmer-30B-GGUF-metrics`.

Everything on one RTX 5090

AD-IQ4_XS plus the vision projector plus the speculative drafter fits in 24.6 GB. Full multimodal, tool calling, and 1.8x faster generation on a single consumer card.

Measured on one RTX 5090 at -c 16384 -fa on, nine prompts across code, prose and reasoning, greedy sampling.

[!NOTE] Speculative decoding costs nothing in quality. The drafter only proposes tokens; the full model verifies every one of them, and under greedy sampling the output is identical to running without it. What it costs is 5.5 GB of VRAM and 18 % of prompt processing speed, since prefill runs through both models and gains nothing from speculation.

Running it

Build from master:

git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON
cmake --build build -j --target llama-server llama-mtmd-cli

[!WARNING] --jinja is required. Without it neither the chat template nor tool-call parsing works.

llama-server -m Muse-Glimmer-30B-AD-IQ4_XS.gguf \
    --mmproj mmproj-Muse-Glimmer-30B-BF16.gguf \
    --jinja -ngl 99 -c 16384 -fa on \
    --temp 1.0 --top-p 0.95 --top-k 64

With speculative decoding:

llama-server -m Muse-Glimmer-30B-AD-IQ4_XS.gguf \
    -md dflash-Muse-Glimmer-30B-BF16.gguf \
    --spec-type draft-dflash --spec-draft-n-max 15 \
    --mmproj mmproj-Muse-Glimmer-30B-BF16.gguf \
    --jinja -ngl 99 -c 16384 -fa on

Reasoning depth is set by a line in the system prompt, not a flag: Reasoning strength: low|medium|high|xhigh. Default is high — at that setting short answers can spend their entire token budget inside the reasoning block, so raise max_tokens or drop to low for quick replies.

Quality

Reference is our own BF16 GGUF, verified bit-identical to the original weights by UUID and all 731 tensor hashes. Two corpora: neutral text in 30 languages, and agentic dialogue in the model's native chat markup with tool calls. Lower KLD is better.

[!NOTE] The same file scores an order of magnitude higher KLD on agentic dialogue than on neutral text. Agentic text is far more predictable (base PPL 2.45 vs 5.46), so the model is very confident and small probability shifts produce large divergence. A KLD number without its corpus is meaningless.

Against another publisher

Twelve builds from another publisher were downloaded and measured against the same BF16 base, with the same binary, on the same hardware.

Below 16 GB our curve sits below theirs everywhere, by 12 % to 41 % depending on the size and corpus. Two of our files win outright — smaller and more accurate:

Above 19 GB theirs is better, by 13 % to 18 %. Their high rungs raise token_embd and leave the FFN at the base type; ours does more or less the opposite. The layout study below explains why that is the right call at 2–4 bits and the wrong one at 6–8.

How the layout was chosen

The first attempt, and why it failed

The first release raised attn_k/v to q80, `attnq/output/gate one step, and both tokenembd` and `output`, leaving the FFN at the base type. It beat the matching llama.cpp preset at every rung — 22 % lower KLD at Q4K_M — but below 15 GB it sat above the preset curve once size was accounted for.

The reason is arithmetic. FFN is 20.7 of 27.9 billion parameters, 74 % of the model. The whole extra budget had gone to the other 26 %.

Measuring what a gigabyte buys

Seven builds from one IQ3_XXS base, each differing by exactly one override. Recovered KL divergence divided by added size gives a price per gigabyte.

output is the worst investment in the model and the first layout raised it at every rung. An earlier 8-bit study had suggested it mattered for top-1 agreement; at 8 bits everything is saturated and that signal was noise.

attn_k and attn_v are the bargain. GQA 32/2 makes them 6656 x 256 — together 0.64 % of the weights. Holding both at q8_0 costs 110 MB.

The optimum moves with bit width

That table is measured at three bits, and it does not transfer upward. At 8 bits the same experiment gives the opposite answer for token_embd: raising it to BF16 takes Q8_0 from 0.000861 to 0.000838 on neutral text and from 0.019968 to 0.018397 on agentic — a 7.9 % improvement, the largest single lever at that size.

The mechanism is straightforward. At low bit widths the FFN is coarse and every bit spent there pays. At high bit widths it is saturated, and the residual error collects in token_embd — the one tensor that is quantized blind, because it is a row lookup rather than a matrix multiply and the importance collector never sees it.

So the recipe is not one rule across the ladder:

• —IQ2 through IQ4_XS — raise all three FFN matrices one step, hold attn_k/v at q8_0, leave everything else alone.
• —Q4_K_M and above — the same, plus token_embd one step.

llama-quantize --imatrix muse-glimmer.imatrix.gguf \
    --tensor-type attn_k=q8_0 --tensor-type attn_v=q8_0 \
    --tensor-type ffn_gate=q4_K --tensor-type ffn_up=q4_K --tensor-type ffn_down=q4_K \
    Muse-Glimmer-30B-BF16-00001-of-00002.gguf out.gguf IQ4_XS 96

Base type and FFN type move together: IQ2XXS/iq2s, IQ2M/iq3xxs, IQ3XXS/iq3s, IQ3M/iq4xs, IQ4XS/q4K, Q4KM/q5K + embd q6K, Q5KM/q6K + embd q80, Q6K/q80 + embd q8_0.

What did not work

Per-layer bit allocation. Importance concentration in ffn_down varies enormously with depth — ten channels out of 19 968 hold 1.6 % of the total at layer 6 and 60 % at layer 42. Raising only the concentrated layers 36–47 versus raising all 52 uniformly gives builds within 1.6 % of each other, in both directions depending on corpus. Indistinguishable. Control published as experiments/…-IQ4_XS_FLAT.gguf.

Beating the presets on layout alone. The control builds carry the same importance matrix as ours and differ only in layout. Against them our advantage is inconsistent — better at some sizes, worse at others. Our lead over the other publisher comes largely from the calibration, not from the bit layout. The controls are published so anyone can check that.

Calibration

imatrix/muse-glimmer.imatrix.gguf is published so anyone can rebuild these files or build their own on the same calibration.

An importance matrix records, for every weight matrix, the mean squared activation arriving on each of its input channels. The quantizer uses those numbers to weight reconstruction error when it searches for block scales: error on a channel that usually carries large activations is penalized more than error on a channel usually near zero.

Both flags are off by default and both matter. Without --parse-special the chat control tokens are tokenized as literal punctuation, so 40 % of the corpus would be calibrated on sequences the model never produces. Without --process-output, output.weight — 1.34 B parameters sitting directly under the logit softcap — gets no importance data at all.

What the calibration shows

Muse Glimmer has 52 layers with 8 collected matrices each, plus the output matrix, for 417 importance vectors. They reduce to five distinct profiles per layer: attn_q, attn_k, attn_v and attn_gate all read the output of attn_norm, so their vectors are identical to the last digit; ffn_gate and ffn_up likewise share ffn_pre_norm. These groups cannot be told apart by importance, only by cost — which is why attn_k/v are raised and attn_q/gate are not.

Y lists the five profiles; X is the share of a tensor's importance held by its 10 strongest input channels and by its strongest 1 %, averaged over 52 layers. Ten channels out of 6656 would hold 0.15 % under a uniform distribution; on the attention input they hold 38.8 %, a factor of 259. For ffn_down the ten strongest of 19 968 hold 19.8 % against a uniform 0.05 %, a factor of 396.

X is the layer index, Y the share of ffn_down importance in its 10 strongest channels. Layers 5–11 sit between 1.6 % and 3.6 %; layers 41 and 42 reach 59 % and 60 %.

K-quants assign one scale to a block of 32 adjacent weights along the input dimension. A single dominant channel forces that scale upward and costs precision on the other 31, so concentration makes a block harder to quantize, not easier. It marks where calibration has the most leverage — but as the per-layer control above shows, that does not translate into a usable rule for moving bits between layers.

One note on attn_gate. Gated attention is new to this architecture and adds a 6656 x 4096 matrix per layer, roughly a third of the attention parameters. No published quantization work has had importance data for it. Its profile is identical to attn_q, attn_k and attn_v, so it cannot be distinguished from them by importance alone.

Reproducing it

llama-imatrix -m Muse-Glimmer-30B-BF16-00001-of-00002.gguf \
    -f calib_train.txt -o part.gguf \
    -c 8192 -b 2048 -ngl 99 \
    --parse-special --process-output --no-ppl

--no-ppl is required at long context: the perplexity path allocates n_ctx x n_vocab x 4 bytes for logits, which is 26.5 GB at 32768 context with this model's 202 048 token vocabulary. Perplexity is not used by importance collection, so disabling it costs nothing.

Merging is exact addition of sums and counts, so the corpus can be split across machines and recombined:

llama-imatrix -m any-model.gguf -c 8192 \
    --in-file part-00.gguf,part-01.gguf,part-long.gguf \
    -o muse-glimmer.imatrix.gguf

Corpus: `AtomicChat/calib-corpora`. Per-tensor statistics: imatrix/imatrix-stats.txt in the metrics dataset.

Credits

Model by Meta Superintelligence Lab. llama.cpp support by @pcuenca and the ggml-org reviewers.