CoolFace
Modelpublic

Abiray/Muse-Glimmer-30B-GGUF

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face
11likes1.3kdownloads
Model Card

Muse-Glimmer-30B - GGUF Quants

This repository contains GGUF quants for [meta-models/Muse-Glimmer-30B](https://huggingface.co/meta-models/Muse-Glimmer-30B), created using llama.cpp. Muse Glimmer is a 30-billion-parameter multimodal model distilled from Muse Spark, purpose-built for autonomous agentic workflows, long-horizon multi-step reasoning, SWE-bench coding tasks, and reliable function calling on consumer hardware.

File Availability & Recommended Hardware

To run vision inputs, download one core model file (.gguf) along with one multimodal vision projector (mmproj-*.gguf).

Core Model Files

File NameSizeQuantizationRec. Memory / VRAMDescription
Muse-Glimmer-30B-Q8_0.gguf29.6 GBQ8_032 GB - 48 GBMaximum precision. Virtually lossless retention compared to BF16.
Muse-Glimmer-30B-Q6_K.gguf22.9 GBQ6_K28 GB - 32 GBNear-lossless output precision. Great for 32GB system/VRAM setup.
Muse-Glimmer-30B-Q5_K_M.gguf19.8 GBQ5KM24 GBHigh Quality balance. Ideal fit for GPUs with 24GB VRAM (e.g., RTX 3090/4090/5090).
Muse-Glimmer-30B-Q4_K_M.gguf16.9 GBQ4KM20 GB - 24 GBRecommended Sweet Spot. Optimal trade-off between speed, memory, and reasoning capacity.
Muse-Glimmer-30B-IQ4_NL.gguf16.1 GBIQ4_NL20 GBNon-Linear 4-bit importance matrix quantization. Strong performance under 17GB.
Muse-Glimmer-30B-IQ4_XS.gguf15.3 GBIQ4_XS18 GB - 20 GBExtra-small 4-bit iQuant for constrained VRAM environments.
Muse-Glimmer-30B-Q3_K_M.gguf14.0 GBQ3KM16 GB - 18 GBStandard 3-bit K-quant. Good option for 16GB VRAM cards.
Muse-Glimmer-30B-IQ3_M.gguf13.1 GBIQ3_M16 GB3-bit medium importance quant with better reasoning recovery than baseline Q3.
Muse-Glimmer-30B-IQ3_XS.gguf12.3 GBIQ3_XS14 GB - 16 GB3-bit extra-small iQuant for lower memory targets.
Muse-Glimmer-30B-IQ3_XXS.gguf11.5 GBIQ3_XXS12 GB - 16 GBHighly compressed 3-bit iQuant. Fits tight memory budgets.

Vision Projector Files (mmproj)

File NameSizePrecisionUsage
mmproj-Muse-Glimmer-30B-BF16.gguf3.85 GBBF16Full-precision ~1.8B ViT perception projector for maximum image fidelity.
mmproj-Muse-Glimmer-30B-Q8_0.gguf2.05 GBQ8_0Recommended. 8-bit quantized vision projector preserving high image understanding at nearly half the RAM.

Quickstart & Usage

1. llama.cpp CLI (With Vision Support)

To run the model with multimodal vision capability:

bash
# Start server with vision support
llama-server \
  -m Muse-Glimmer-30B-Q4_K_M.gguf \
  --mmproj mmproj-Muse-Glimmer-30B-Q8_0.gguf \
  -c 131072 \
  --port 8080