AntimatterAI/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-MLX-5bit

<div align="center">

Qwen3.6-35B-A3B — Claude Opus 4.7 Reasoning

MLX 5-bit &nbsp;|&nbsp; Apple Silicon Optimized

Converted & released by [Antimatter AI](https://www.antimatterai.com)

Original model by [lordx64](https://huggingface.co/lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled)

35B total &nbsp;→&nbsp; ~3B active per token &nbsp;|&nbsp; 5.5 bpw &nbsp;|&nbsp; ~22 GB on disk &nbsp;|&nbsp; runs on 32 GB+ Apple Silicon

</div>

What is this?

An MLX-native 5-bit quantization of lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled — a Mixture-of-Experts reasoning model fine-tuned to think like Claude Opus 4.7.

The original bf16 weights (~70 GB) are too large for most local setups. This conversion brings the model to ~22 GB with 5-bit affine quantization via Apple's MLX framework, making it practical to run entirely on-device on a Mac with 32 GB or more of unified memory.

Why this model matters

Quick Start

Install

pip install mlx-lm

Generate

from mlx_lm import load, generate

model, tokenizer = load("AntimatterAI/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-MLX-5bit")

messages = [{"role": "user", "content": "How many positive integers less than 1000 have digits that sum to 20?"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)

response = generate(model, tokenizer, prompt=prompt, max_tokens=8192, verbose=True)
print(response)

Serve (OpenAI-compatible API)

python -m mlx_lm server \
  --model AntimatterAI/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-MLX-5bit \
  --host 127.0.0.1 \
  --port 8080

Then query it like any OpenAI endpoint:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "AntimatterAI/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-MLX-5bit",
    "messages": [{"role": "user", "content": "Explain the Riemann hypothesis in simple terms."}],
    "max_tokens": 4096
  }'

Architecture

Quantization Details

Router gates are kept at 8-bit precision to preserve expert routing quality — the gates are small tensors but critical for MoE performance.

Benchmarks

Benchmarks below are from the original bf16 model as evaluated by [lordx64](https://huggingface.co/lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled). Quantization to 5-bit typically incurs minimal degradation on MoE architectures due to the sparse activation pattern.

MMLU-Pro subject breakdown:

Full evaluation data: lordx64/qwen3-6-distill-evals

Memory & Performance Guide

The MoE architecture activates only ~3B parameters per token, so prefill memory per step is much smaller than a dense 35B model. Expect ~30–50 tokens/sec decode speed on M-series chips depending on memory bandwidth.

Reasoning Behavior

This model emits explicit <think>…</think> reasoning blocks before its final answer — this is by design, inherited from the Claude Opus 4.7 distillation. Example:

<think>
I need to count positive integers less than 1000 whose digits sum to 20.
Let me denote a 3-digit number as having digits a, b, c where...
[detailed step-by-step reasoning]
</think>

The answer is 75.

For applications where you only need the final answer, strip everything between <think> and </think> in post-processing.

Preventing Thought Loops

At 5-bit quantization, this model will degenerate inside <think> blocks and during extended generation. The degeneration takes multiple forms: paragraph-level repetition, single-word stutter, synonym chain runaway, word concatenation, and free-association word salad. Repetition penalties alone are insufficient — you must also disable extended thinking and cap token output.

Recommended: disable thinking mode at the server level:

python -m mlx_lm server \
  --model /path/to/model \
  --chat-template-args '{"enable_thinking": false}'

Recommended generation parameters:

{
  "repetition_penalty": 2.0,
  "repetition_context_size": 1024,
  "temperature": 0.7,
  "max_tokens": 2048
}

For production use, the included proxy (mlx_openai_input_limit_proxy.py) provides output-side loop detection that catches degeneration patterns and truncates responses at the last coherent sentence. This is the most reliable defense.

Why this happens: The 5-bit quantization reduces the model's ability to maintain coherent long-range generation. Combined with reasoning-distillation training that strongly biases toward extended output, the model enters self-reinforcing degeneration modes that escalate from word repetition to increasingly creative forms of incoherence.

Training Lineage

Qwen/Qwen3.6-35B-A3B (Apache 2.0)
  └── lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled (LoRA SFT on Claude Opus 4.7 traces)
        └── AntimatterAI/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-MLX-5bit (this model — MLX 5-bit conversion)

Limitations

• —Reasoning over knowledge. Distillation transfers how to think, not new facts. The model's knowledge boundary is that of Qwen 3.6.
• —Long generation budgets. The model will use 5–30K tokens of <think> reasoning on hard problems. Set max_tokens accordingly.
• —Attention-only LoRA. Expert FFNs are untouched from the base — domains where Claude and Qwen diverge may show uneven results.
• —Distillation provenance. Training data was generated via Anthropic's Claude API. Downstream users should confirm compliance with Anthropic's usage policies.

Citation

If you use this model, please cite the original work and this conversion:

@misc{lordx64_qwen36_distill_2026,
  title  = {Qwen3.6-35B-A3B distilled from Claude Opus 4.7 reasoning},
  author = {lordx64},
  year   = {2026},
  url    = {https://huggingface.co/lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled},
}

@misc{qwen36_a3b_2026,
  title  = {Qwen3.6-35B-A3B},
  author = {Qwen Team},
  year   = {2026},
  url    = {https://huggingface.co/Qwen/Qwen3.6-35B-A3B},
}

@misc{antimatterai_mlx_5bit_2026,
  title  = {Qwen3.6-35B-A3B Claude Opus 4.7 Reasoning — MLX 5-bit conversion},
  author = {Antimatter AI},
  year   = {2026},
  url    = {https://huggingface.co/AntimatterAI/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-MLX-5bit},
}

Acknowledgements

• —[lordx64](https://huggingface.co/lordx64) — original model author; fine-tuned Qwen3.6 on Claude Opus 4.7 reasoning traces and published the weights under Apache 2.0.
• —[Qwen Team](https://huggingface.co/Qwen) — for the Qwen3.6-35B-A3B base model with a permissive license.
• —[Anthropic](https://www.anthropic.com) — for Claude Opus 4.7, the teacher model.
• —[Apple MLX Team](https://github.com/ml-explore/mlx) — for the MLX framework enabling efficient on-device inference.
• —[Unsloth](https://github.com/unslothai/unsloth) — for accelerated LoRA training tooling used in the original fine-tune.

<div align="center">

[Antimatter AI](https://www.antimatterai.com) — Building Digital Solutions That Matter

AI Development &nbsp;|&nbsp; Product Design &nbsp;|&nbsp; Healthcare Apps &nbsp;|&nbsp; IoT

</div>