Modelpublic

andrevp/Qwen3.6-35B-A3B-3bit-MLX

sourceHugging Faceapache-2.0updated 5mo agoView on Hugging Face

4likes340downloads

Model Card

Qwen3.6-35B-A3B 3-bit MLX

3-bit quantized version of Qwen3.6-35B-A3B optimized for Apple Silicon inference via MLX.

Model Details

—Base Model: Qwen/Qwen3.6-35B-A3B
—Quantization: 3-bit (MLX format)
—Model Size: ~14 GB
—Architecture: Hybrid MoE (Mixture of Experts) + Mamba
—10 full-attention layers
—30 linear-attention (Mamba) layers — O(1) state per token
—Total Parameters: 35B (3B active per token)
—Context Length: 262,144 tokens (256K)
—Vocabulary: 248,320 tokens

Performance (M4 Pro 24GB)

Metric	Value
Decode Speed	~62 tok/s
Prefill Speed	~400-540 tok/s
Max Context (24GB RAM)	87,500 tokens
RAM Usage	~15-16 GB

Benchmark (Thinking OFF)

Benchmark	Score
InstructFollow-15	75/100
ReasonMath-15	72/100
DataExtract-15	90/100

Comparison vs Q2KXL GGUF (same hardware)

Metric	Q2_K_XL	3-bit MLX
Decode Speed	42 tok/s	62 tok/s
Max Context	~8K	87.5K
Model Size	11.4 GB	~14 GB

Usage

python

from mlx_lm import load, generate

model, tokenizer = load("andrevp/Qwen3.6-35B-A3B-3bit-MLX")
response = generate(model, tokenizer, prompt="Hello!", verbose=True)

Disable Thinking Mode

python

response = generate(
    model, tokenizer,
    prompt="Hello!",
    max_tokens=512,
    temp=0.0,
)

For chat template with thinking disabled, pass chat_template_kwargs={"enable_thinking": False} when applying the tokenizer.

Requirements

—Hardware: Apple Silicon Mac (M1/M2/M3/M4)
—RAM: 16 GB minimum, 24 GB recommended for long context
—Python: 3.11+
—MLX: 0.31+
—mlx-lm: latest

License

Apache 2.0 (same as base model)