Schackay3/Qwen3-Next-80B-A3B-Instruct-ternary-GGUF

Qwen3-Next-80B-A3B-Instruct — Ternary (1.58-bit expert) GGUF

To our knowledge, this is the first publicly released ternarized Mixture-of-Experts model. All prior published ternary conversions (PT-BitNet, PT2-LLM, Falcon3-1.58bit, Llama3-8B-1.58) target dense models.

Every one of the model's 24,576 expert weight matrices (512 experts x 48 layers, gate/up/down projections) was quantized to ternary values {-1, 0, +1} with per-block scale and offset, using GPTQ-style error- compensating reconstruction. The conversion ran for 34 hours on a single NVIDIA Quadro RTX 4000 (8GB) in a 2016-era dual-Xeon workstation.

What was quantized

The trunk was deliberately left at full precision: it is a small fraction of total parameters and by far the riskiest part to quantize aggressively.

Measured results

Reference machine: 2x Intel Xeon E5-2697 v4 (2016, no AVX-512, no AMX), 256GB DDR4-1866, Quadro RTX 4000 8GB. llama.cpp b10331, flags -ngl 999 -ncmoe 999 -fa 1 -t 18 (attention on GPU, experts in system RAM).

Quality (wikitext2 perplexity, 2048 ctx)

Speed & size (vs the stock Q4KM release of the same model)

Coherence (greedy decode)

Q: What are the three primary colors?
A: The three primary colors are red, blue

"The capital of France is" -> "Paris. The capital of the United States"
"def reverse_string(s):"   -> "\n    # Write your code here.\n"

Honest limitations

• —This is meaningfully worse than the original. A +51% perplexity increase is real and you will likely notice it on hard reasoning and long-form code. If you need maximum quality, use the standard Q4KM release.
• —Calibration was wikitext2 only (128 x 2048 tokens) — generic English prose for a model trained on code, math, multilingual and agentic data. Domain- matched calibration would very likely improve results.
• —SSR (Structural Similarity-based Reordering) was not enabled, nor was a distillation teacher used. Both are known to help.
• —The GGUF is Q2_K (2.94 bpw), not a true 1.58-bit format. llama.cpp's native ternary formats (TQ10/TQ20) are symmetric with 256-element blocks; the GPTQ reconstruction produces asymmetric group-128 ternary (measured: exactly 3 distinct values per 128-block, offsets averaging |mu|/alpha = 0.10-0.19). Q2_K stores a scale and min per sub-block and represents this far more faithfully. A native asymmetric group-128 ternary kernel would recover the remaining ~0.9 bpw and push speed toward ~35 tok/s.
• —MTP (multi-token prediction) head is not included (--no-mtp), so speculative decoding is unavailable.
• —Single eval set. No MMLU / HumanEval / GSM8K numbers yet.

Novel finding: expert utilization varies with depth

Experts routed fewer than 32 calibration tokens were left in bf16 rather than quantized on unreliable statistics. The count of such "starved" experts is not uniform across the network:

Later layers concentrate routing on fewer experts. We have not seen this measured elsewhere and it has direct implications for MoE quantization: a uniform calibration budget under-serves deep layers.

Bug found in PT2-LLM

solve_closed_form_alpha_mu() in the upstream PT2-LLM quantizer has its epsilon commented out, so denom reaches exactly 0 for all-zero ternary rows, producing NaN that silently propagates through the entire weight matrix. This is harmless on large dense matrices but fatal on small expert matrices: 484 of 512 experts failed before the fix, 0 after. Fixed by clamping the denominator and applying the AGA fallback per-row instead of per-matrix.

Usage

llama-cli -m qwen3-ternary-Q2_K.gguf -ngl 999 --n-cpu-moe 999 -t 18 -fa on -cnv

Tune -t to your core count; on dual-socket systems, single-socket thread counts often beat using all cores (NUMA contention). On this machine 18 threads was fastest for generation, 24 for prompt processing.

Credits & license

• —Base model: Qwen/Qwen3-Next-80B-A3B-Instruct (Apache 2.0)
• —Quantization method: PT2-LLM (ICLR 2026), adapted for MoE + 8GB VRAM
• —Runtime: llama.cpp

Released under Apache 2.0, matching the base model.