Litwein/Qwen3.6-35B-A3B-Claude-4.7-Opus-oQ4e-DWQ-MC-MTP-Vision-MLX

Qwen3.6-35B-A3B-Claude-4.7-Opus-oQ4e-DWQ-MC-MTP-Vision-MLX

### ⭐ v2 — the best build of this model we ship Two-stage imatrix → DWQ (reasoning, then math/code). Materially better on code than the v1 repos: HumanEval 90.2 % (v1-stage: 84.8 %) · LiveCodeBench 39.9 % (32 %) · MBPP 75.4 %. If you used any of the four …-DWQ4-… / …-oQ4-… v1 repos, use this instead.

A 4-bit MLX quantization of `lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled` — vision + text, with an embedded MTP head for speculative decoding.

⚠️ These are quantized weights. All capability comes from the base model — please star/cite it first. This repo's contribution is the quantization recipe + MTP/vision packaging and its fidelity/benchmark validation.

Model lineage

Qwen/Qwen3.6-35B-A3B                                  (Apache-2.0 base · 35B MoE · 256 experts · ~3B active · 262k ctx)
  └─ lordx64/…-Claude-4.7-Opus-Reasoning-Distilled     (SFT distill of Claude Opus 4.7 reasoning)
       └─ THIS REPO: oQ4e (imatrix) → DWQ¹ reasoning → DWQ² math/code  + MTP + vision

• —Architecture: Qwen3.6-35B-A3B — MoE, 256 experts (8 routed + 1 shared), ~3B active params/token, up to 262,144-token context.
• —Base distillation (by [lordx64](https://huggingface.co/lordx64)): SFT on ~7,800 chain-of-thought traces from Claude Opus 4.7, teaching explicit <think>…</think> reasoning.

Quantization: oQ4e (imatrix) → two-stage DWQ

Not a plain round-to-nearest 4-bit. Three steps:

1. 1.`oQ4e` — imatrix mixed precision. oMLX's enhanced quantizer builds an importance matrix from calibration activations (sized for MoE expert coverage) and allocates bits by sensitivity. Base 4-bit affine, group size 64, with sensitive tensors promoted: 196 → 8-bit, 114 → 5-bit, 1 → 6-bit.
2. 2.DWQ¹ — reasoning. The sub-8-bit affine scales/biases are gradient-distilled toward an `oQ8e` (imatrix 8-bit) teacher on reasoning calibration (KL over the teacher's top-1024 logits, temperature 2.0, Adam, cosine LR, validation early-stopping, finalized from the best checkpoint).
3. 3.DWQ² — math/code. A second distillation pass over a code-heavy mix, cascading from the stage-1 model. This is what lifts HumanEval 84.8 % → 90.2 % and LiveCodeBench 32 % → 39.9 %.

Why two stages: stage 1 recovers general fidelity to the 8-bit teacher; stage 2 re-tunes the 4-bit grid toward code/math activations where the v1 builds were weakest. The reasoning replay partition in stage 2 keeps the reasoning behaviour from drifting.

Evaluation

Measured on this quant (served via oMLX)

Run by the uploader on this 4-bit build via oMLX. Reported to show that the recipe preserves and re-tunes base capability, not as official base-model results — confirm harness/shot settings against your own eval before citing.

Inherited base-model benchmarks (bf16, from the base card)

Reported by the base model via lm-evaluation-harness; a capability reference for the family, not re-measured on this quant.

Note on long-form math. This is a reasoning model that routinely emits 8,000+ tokens of <think> on hard problems. Fixed-budget pass/fail there is token-budget-limited, not quantization-limited. Give it a large max_tokens (32k) for hard problems.

Repos in this family

Both v2 repos share the same distilled backbone + MTP head (byte-identical); they differ only by the vision tower — identical task quality.

How to run

MLX weights (Apple Silicon). Tested serving path is [oMLX](https://omlx.app) ≥ 0.5.0, which supports this model's native MTP speculative decoding and vision out of the box. This repo includes the vision tower (vision_tower.*, bf16) — it accepts images as well as text.

# 1. place the folder in your oMLX models directory
mv Qwen3.6-35B-A3B-Claude-4.7-Opus-oQ4e-DWQ-MC-MTP-Vision-MLX ~/.omlx/models/

# 2. enable MTP once, then call the OpenAI-compatible API
curl -X PUT http://127.0.0.1:8003/admin/api/models/Qwen3.6-35B-A3B-Claude-4.7-Opus-oQ4e-DWQ-MC-MTP-Vision-MLX/settings \
  -H "Authorization: Bearer sk-local" -H "Content-Type: application/json" \
  -d '{"mtp_enabled": true}'

curl -X POST http://127.0.0.1:8003/v1/chat/completions \
  -H "Authorization: Bearer sk-local" -H "Content-Type: application/json" \
  -d '{"model": "Qwen3.6-35B-A3B-Claude-4.7-Opus-oQ4e-DWQ-MC-MTP-Vision-MLX",
       "messages": [{"role": "user", "content": "Write a Python LRU cache with tests."}],
       "max_tokens": 8000, "temperature": 0.6}'

The backbone also loads directly in stock `mlx-lm` as a qwen3_5_moe text model. MTP speculative decoding needs an MTP-aware runtime — oMLX (tested). Note the oMLX model id is case-sensitive (it matches the folder name).

Recommended sampling

temperature 0.6, top_p 0.95, top_k 20, min_p 0. For hard reasoning set max_tokens ≥ 32000 — the model thinks in explicit <think>…</think> blocks. Avoid greedy (temperature 0): reasoning models degenerate there.

Intended use & limitations

• —Built for hard reasoning + code: competition math, STEM, code generation, multi-step logic.
• —Reasoning ≠ knowledge. Quantization and the base distillation transfer how to reason, not new facts.
• —Quantization loss: 4-bit is lossy vs bf16; for maximum quality use the bf16 base or an 8-bit quant.
• —Distillation provenance: the base's traces were generated with Anthropic's Claude Opus 4.7. Confirm compliance with Anthropic's usage policy.

Acknowledgements

• —[lordx64](https://huggingface.co/lordx64) — the reasoning-distilled base this repo quantizes. All capability is theirs.
• —Qwen team — Qwen3.6-35B-A3B (Apache-2.0).
• —Anthropic — Claude Opus 4.7, the reasoning teacher for the base distillation.
• —Apple MLX — mlx, mlx-lm (mlx_lm.quant.dwq), mlx-vlm.
• —oMLX — the oq/oQe imatrix quantizer, MTP + vision serving runtime.
• —Calibration corpora — Open-R1 (math, verified coding), NVIDIA OpenCodeReasoning, BigCode self-oss-instruct, SWE-bench (SWE-smith trajectories).

License

Apache-2.0, inherited from the base model and Qwen3.6-35B-A3B.

Citation

@misc{qwen36_a3b_2026, title={Qwen3.6-35B-A3B}, author={Qwen Team}, year={2026},
  howpublished={\url{https://huggingface.co/Qwen/Qwen3.6-35B-A3B}} }
@misc{lordx64_qwen36_distill_2026,
  title={Qwen3.6-35B-A3B distilled from Claude Opus 4.7 reasoning}, author={lordx64}, year={2026},
  howpublished={\url{https://huggingface.co/lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled}} }