Modelpublic

dxx117/Qwen3.6-35B-REAP-IQ1M

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face

3likes179downloads

Model Card

Qwen 3.6 35B A3B REAP IQ1_M GGUFs

This repo contains three experimental IQ1_M GGUF quants derived from REAP-pruned Qwen 3.6 35B A3B checkpoints. I just wanted to get a local LLM that could answer the car wash test correctly on my iPad Pro with 8GB RAM as all the other smaller LLMs it could run just couldn't, and also faster than 3tk/s for the full 35B IQ1M model due to partial offload to swap.

These are not plain base-model quants. They come from REAP-pruned MoE checkpoints, then were converted to trunk-only GGUF and quantized with a Bartowski-style Qwen 3.6 IQ1_M tensor policy.

What These Are

Starting point:

—Qwen/Qwen3.6-35B-A3B

Pruning method:

—REAP expert pruning on the HF BF16 model

Quantization method:

—GGUF export
—trunk-only / no-mtp conversion path
—fresh per-model imatrix
—Bartowski-style Qwen 3.6 IQ1_M tensor map

Recommended Variant

—Use qwen3.6-35b-reap25_iq1m.gguf if you want the strongest general-purpose REAP variant.
—Use qwen3.6-35b-reap40_iq1m.gguf if you care more about instruction-following and strict prompt obedience.
—Use qwen3.6-35b-reap50_iq1m.gguf only if size is the main priority.

Benchmarks

35B Model	Size	HellaSwag	ARC-C	IFEval Prompt Strict	IFEval Inst Strict	Car wash test	iPad Pro M1 CPU TG
`reap25_iq1m`	7.93 GB	77.5642	50.5017	0.3641	0.4868	Yes	7.18
`reap40_iq1m`	6.68 GB	71.2209	49.4983	0.4067	0.5432	Sometimes	8.06
`reap50_iq1m`	5.82 GB	64.2004	50.1672	0.4011	0.5360	No	7.91
`bartowski_iq1m`	10.04 GB	80.1733	50.1672	0.4584	0.5707	Yes	3-4
`unsloth_q4_k_xl`	20.82 GB	82.4935	53.1773	0.3272	0.4616	Yes	N/A

[image] [image]

Interpretation

—reap25 is the strongest broad-quality REAP quant.
—reap40 is the most interesting instruction-following variant. It gives up a lot on HellaSwag, but beats reap25 and reap50 on both strict IFEval metrics.
—reap50 is the most compressed and the most uneven. It keeps ARC surprisingly well, but broad completion quality drops much harder.

One likely explanation is that heavier REAP pruning narrows the model more than it destroys literal instruction compliance. That can hurt broad reasoning / continuation tasks while still preserving, or sometimes slightly improving, rigid format-following behavior.