yogeshjog/Muse-Glimmer-30B-Q2_K-GGUF

Muse-Glimmer-30B Q2_K GGUF

An unofficial Q2_K GGUF quantization of meta-models/Muse-Glimmer-30B.

This quantization was created directly from the official BF16 weights using llama.cpp.

The goal is to make Muse Glimmer 30B practical on lower-memory systems, particularly Apple Silicon machines, while retaining as much capability as possible.

Note: Q2_K is an aggressive quantization. Expect some quality degradation compared with BF16 and higher-bit quantizations such as Q4, Q5, or Q8.

Model Details

Quantization Process

This model was not requantized from an existing Q4/Q8 model.

It was generated directly from the official Hugging Face BF16 weights:

meta-models/Muse-Glimmer-30B
            │
            ▼
    Official BF16 weights
            │
            ▼
     BF16 GGUF (~53 GiB)
            │
            ▼
     llama.cpp Q2_K
            │
            ▼
     Q2_K GGUF (9.95 GiB)

The conversion was performed using my open-source quantization utility:

phygineer/quantize

The underlying conversion and quantization are performed using llama.cpp.

Quantization output

llama_model_quantize_impl: model size  = 53131.48 MiB (16.00 BPW)
llama_model_quantize_impl: quant size  = 10179.12 MiB (3.07 BPW)

llama_quantize: quantize time = 79786.76 ms
llama_quantize: total time    = 79786.76 ms

Tested Hardware

This quantization was created and tested locally on:

Apple MacBook Pro
Apple M3 Max
36 GB unified memory

No cloud GPU was required for the conversion or quantization.

The BF16 GGUF conversion produced approximately 55.7 GB of output and completed successfully despite the machine having 36 GB of unified memory.

The Q2_K quantization itself completed in approximately 80 seconds.

Ollama Test

The resulting GGUF was successfully imported into Ollama and tested with a configured context length of 32,768 tokens.

NAME                      SIZE     PROCESSOR    CONTEXT
muse-glimmer-q2:latest    11 GB    100% GPU     32768

Test environment:

• —Apple M3 Max
• —36 GB unified memory
• —32K configured context
• —100% GPU offload reported by Ollama

A basic generation test completed successfully.

The SIZE reported by Ollama should not be interpreted as guaranteed total system memory consumption under a fully populated 32K context. Actual memory usage depends on context utilization, KV cache configuration, runtime buffers, and Ollama/llama.cpp versions.

Running with Ollama

Download:

hf download yogeshjog/Muse-Glimmer-30B-Q2_K-GGUF \
  Muse-Glimmer-30B-Q2_K.gguf \
  --local-dir .

Create a file named Modelfile:

FROM ./Muse-Glimmer-30B-Q2_K.gguf

PARAMETER num_ctx 32768

Import it:

ollama create muse-glimmer-q2 -f Modelfile

Run:

ollama run muse-glimmer-q2

Check the loaded model:

ollama ps

Running with llama.cpp

Using a recent version of llama.cpp with Muse Glimmer support:

llama-cli \
  -m Muse-Glimmer-30B-Q2_K.gguf \
  -c 32768

Why Q2_K?

Muse Glimmer 30B is a relatively large model for consumer hardware.

Higher-bit quantizations provide better quality but require considerably more memory.

This Q2_K version reduces the model to approximately:

9.95 GiB
3.07 bits per weight

This makes it interesting for systems such as:

• —Apple Silicon Macs
• —16–24 GB GPU systems
• —laptops and workstations with limited unified memory
• —experimentation with larger context windows
• —local inference where memory usage matters more than maximum model quality

The objective is not to claim that Q2_K matches the original model.

Instead, this release explores:

How much of Muse Glimmer 30B's capability can be retained at roughly 10 GiB?

Quality Warning

Q2_K is a very aggressive quantization.

It may negatively affect:

• —reasoning
• —coding accuracy
• —instruction following
• —factual recall
• —structured output
• —multimodal performance
• —long-context performance

For workloads where quality is more important than memory consumption, consider a higher-bit quantization.

This model should therefore be considered an experimental low-memory build.

More extensive benchmarking is planned.

Reproducing the Quantization

The quantization can be reproduced using:

phygineer/quantize

Source model:

meta-models/Muse-Glimmer-30B

Conversion:

Hugging Face BF16
        ↓
BF16 GGUF

Quantization:

BF16 GGUF
        ↓
llama-quantize Q2_K
        ↓
Muse-Glimmer-30B-Q2_K.gguf

This approach avoids requantizing an already quantized checkpoint.

Original Model

This repository contains only an unofficial quantized version.

For architecture details, intended use, limitations, safety information, and authoritative documentation, refer to the original model:

meta-models/Muse-Glimmer-30B

License

The base model is released under the Apache License 2.0.

Users of this quantization should review and comply with the license and any usage requirements of the original model.

Credits

Original model: Meta Superintelligence Labs Base model: meta-models/Muse-Glimmer-30B GGUF / quantization: llama.cpp Quantization pipeline: phygineer/quantize Quantization: Q2_K Published by: yogeshjog

Disclaimer

This is an unofficial community quantization and is not affiliated with or endorsed by Meta.

Low-bit quantization changes model weights and may materially affect model behavior and output quality. Evaluate the model for your workload before relying on its outputs.