Modelpublic

zSaiko/Pantheon-Reasoning-26B-A4B-1.1-V2-APEX-GGUF

sourceHugging Faceapache-2.0updated 2d agoView on Hugging Face

0likes1.4kdownloads

Model Card

Pantheon-Reasoning-26B-A4B-1.1-V2 — APEX GGUF

Mixed-precision APEX GGUF quantizations of Gryphe/Pantheon-Reasoning-26B-A4B-1.1-V2.

These quants use a Pantheon-specific importance matrix and per-tensor mixed precision rather than a uniform GGUF quantization scheme.

SORRY, Im pretty sure this quantization is somewhat shit after some personal use? If not just comment BUT I basically went off and expected a quantization recipe of StyleTuneV2 to work well here but that model only had lm_head trained and this is vastly differently trained it seems so that was pretty stupid :P - sorry, my first quant

Available variants

Variant	Description
APEX iQuality	Higher-fidelity mixed-precision quant. Approximately 5.89 BPW.
APEX iQuality — Q8 LM Head	Same iQuality tensor allocation, except `output.weight` is stored at Q8_0 for testing LM-head precision.
APEX iCompact	Smaller mixed-precision variant at approximately 4.31 BPW.

Because APEX uses per-tensor mixed precision, the standard quant type at the end of each filename is an approximate size/BPW class rather than a description of every tensor in the file.

Fidelity benchmark

Measured against BF16 reference logits on WikiText-2 using llama.cpp, 128 × 512-token chunks (32,640 scored next-token positions):

Variant	Mean KLD	Same top token	RMS Δp
APEX iQuality	0.861731	72.387%	8.638%
APEX iQuality — Q8 LM Head	0.856834	72.540%	8.636%
APEX iCompact	1.386163	61.759%	11.638%

The Q8 LM-head experiment produced only a small aggregate improvement over the standard iQuality variant.

Notes

Gemma 4 contains tensor dimensions that are incompatible with some requested K-quant formats. In particular, ffn_down_exps.weight has a 704-wide dimension, so llama.cpp may use compatible fallback formats when necessary.

Observed examples include:

—requested IQ4_XS → stored as IQ4_NL
—requested Q5_K → stored as Q5_1
—requested Q6_K → stored as Q8_0

These are expected shape-dependent fallbacks.

Base model

See the original model card for architecture, training details, intended usage, and model-specific notes:

Gryphe/Pantheon-Reasoning-26B-A4B-1.1-V2

Credits

Original model by Gryphe.

APEX mixed-precision methodology/configuration derived from the work by EmanuelOverride on Gemma 4 APEX GGUF quantization.