CoolFace
Modelpublic

hyperspaceai/Qwen3.6-35B-A3B-asym-2bitexp-GGUF

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face
0likes52downloads
Model Card

Qwen3.6-35B-A3B — Asymmetric 2-bit-Expert GGUF (16 GB Mac)

An antirez/ds4-style asymmetric mixed-precision quant of Qwen3.6-35B-A3B (35B total / ~3.3B active, 256 routed experts / 8 active), built for the Hyperspace network so the model fits and runs on a 16 GB unified-memory Mac.

The MoE insight: ~86–93% of the parameters live in the routed experts, but only 8 of 256 are active per token. So we put the lowest bits on the routed-expert bulk and protect the always-active path (shared expert, attention, embeddings, output head) at higher precision — imatrix-calibrated.

Quantization scheme

Tensor groupShareTypebpw
ffn_gate_exps, ffn_up_exps (routed)~52%IQ2_S2.56
ffn_down_exps (routed, more sensitive)~35%IQ3_S3.44
ffn_*_shexp (shared expert, active every token)~1%Q6_K6.56
attn_*~5%Q4_K4.50
token_embd, output~6%Q6_K6.56

Overall 3.06 BPW, 13.25 GB on disk. imatrix calibrated on the standard calibration_datav3 corpus (129 chunks).

Why this fits 16 GB

IQ4_XS baseline**this asym quant**
Size on disk18.6 GB13.25 GB (−29%)
16 GB Mac @ 16k ctx (q8_0 KV)❌ does not fit✅ ~13.8 GiB, fits with headroom
Wikitext PPL (ctx 512)6.89457.2846 (+5.66%)

The +5.66% perplexity is the honest cost of 2-bit routed experts — small for an IQ2-class quant, because the asymmetry protects the dense/always-active path. Verified coherent on coding, reasoning (bat-and-ball), and explanation prompts.

Files

  • —Qwen3.6-35B-A3B-asym-2bitexp.gguf — the model (13.25 GB)
  • —imatrix.dat — the importance matrix used (for reproducibility / requant)

Notes / limitations

  • —The MTP draft layer is not present (dropped during conversion); served as a standard autoregressive model.
  • —Architecture qwen35moe, GQA attention (head_count_kv=2, head_dim=256), 40 layers, 262144 max context.
  • —Best served on Apple Silicon via the Hyperspace prism engine or llama.cpp; on a 16 GB Mac use q8_0 KV cache and ≤16k context.

Built and validated on an H100 (perplexity, serving). Quality cost and fit math are measured, not estimated.