wangkezun/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-VLM-MLX-oQ6

Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-VLM-MLX-oQ6

Mixed-precision quantization for Apple Silicon, with vision-language (VLM) capabilities preserved. The recommended sweet-spot version — best quality-to-size ratio in the series.

Quantized from `lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled` using oMLX's oQ6 algorithm (sensitivity-aware mixed-precision quantization).

📊 Specs

🚀 Quick Start

Install

pip install mlx-vlm
# Or with uv:
uv tool install mlx-vlm --with torch --with torchvision

Inference (image + text)

mlx_vlm.generate \
  --model wangkezun/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-VLM-MLX-oQ6 \
  --image /path/to/image.jpg \
  --prompt "Describe this image in detail." \
  --max-tokens 256

Python API

from mlx_vlm import load, generate

model, processor = load(
    "wangkezun/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-VLM-MLX-oQ6"
)

output = generate(
    model,
    processor,
    image="/path/to/image.jpg",
    prompt="What's in this image?",
    max_tokens=512,
)
print(output)

OpenAI-compatible Server

mlx_vlm.server \
  --model wangkezun/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-VLM-MLX-oQ6 \
  --port 8080

Drop-in compatible with OpenAI clients including AstrBot, Open WebUI, LibreChat, and Continue.dev.

📈 Measured Performance

Benchmarked on MacBook Pro M5 Max 128GB:

Quality vs oQ4

The jump from oQ4 to oQ6 yields a substantial knowledge-recall improvement. Tested on the same image input:

• —oQ4: "A blonde man with a bright smile..."
• —oQ6: "Actor Antony Starr in his role as Homelander from The Boys (TV series)..."

oQ6 retains specific entity-level knowledge (actor names, show titles, character names) that oQ4 collapses into generic descriptions. This is the threshold where world knowledge survives quantization meaningfully.

🧠 Model Behavior

Inherits the Claude reasoning distillation: the model uses <think>...</think> tags to structure its chain-of-thought before producing the final response.

Best for:

• —Multimodal reasoning tasks (image analysis with complex thinking)
• —Agentic workflows / tool use
• —Scenarios where visible reasoning is desired
• —Knowledge-intensive image understanding (named entity recognition, cultural references)

🔬 Quantization Details

• —Source model: Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled (BF16 MLX-converted)
• —Sensitivity model: 8-bit quantization of the same distilled model (self-referenced sens for tight distribution alignment)
• —Non-quant weight dtype: bfloat16 (M3+ optimal)
• —Text-Only mode: OFF (vision tower preserved)
• —Quantizer: oMLX with mlx-vlm conversion path

The vision processor configurations (preprocessor_config.json, video_preprocessor_config.json, processor_config.json) were sourced from the official Qwen base model to ensure proper image input handling — these were missing from the upstream distilled checkpoint.

📦 Other Versions in This Series

Choosing a version:

• —Text-only workflows (coding, agents, dialogue) → Text variants are faster and lighter
• —Image input needed (OCR, visual analysis, screenshot understanding) → VLM variants
• —oQ6 is the sweet spot for most use cases. oQ8 yields diminishing returns relative to its size.

⚠️ Disclaimer

This model derives from a chain of upstream work:

1. 1.Base model `Qwen/Qwen3.6-35B-A3B` by Alibaba's Qwen team (Apache-2.0)
2. 2.Distilled variant `lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled` by lordx64 using Claude 4.7 Opus reasoning outputs (Apache-2.0)
3. 3.This quantization by @wangkezun using oMLX on Apple Silicon

This model is not affiliated with or endorsed by Anthropic, PBC. "Claude" is a trademark of Anthropic, PBC. The use of "Claude" in this model name is purely descriptive (nominative fair use) to indicate the upstream training data lineage.

By using this model, you agree to comply with:

• —The Apache-2.0 license inherited from the base model
• —Any applicable license terms of the upstream distillation dataset
• —Local laws and regulations governing AI model usage in your jurisdiction

🙏 Acknowledgments

• —Alibaba Qwen Team — for the Qwen3.6-35B-A3B base model
• —lordx64 — for the reasoning-focused LoRA distillation
• —Jundot (oMLX team) — for the oQ mixed-precision quantization algorithm
• —Apple MLX team — for the MLX framework and tooling
• —mlx-vlm contributors — for the VLM conversion path

📜 License

Apache-2.0 (inherited from base model).

Generated: 2026-04-26 Quantizer: @wangkezun