wang-yang/Agents-A1-MTPLX-Q4

Agents-A1-MTPLX-Q4

4-bit quantized MLX version of InternScience/Agents-A1 with grafted MTP (Multi-Token Prediction) head for speculative decoding on Apple Silicon.

Model Details

• —Base model: InternScience/Agents-A1 (Qwen3.5-MoE architecture, 35B total / 3B active parameters)
• —Quantization: 4-bit affine (group size 64), router gates at 8-bit
• —MTP head: Grafted from Qwen3.5-35B-A3B (4-bit quantized, 1 layer)
• —Format: MLX safetensors
• —Disk size: ~18 GB (model) + 1.6 GB (MTP sidecar)

Architecture

• —Hidden size: 2048
• —Layers: 40 (hybrid linear + full attention)
• —Experts: 256 total, 8 active per token
• —Vocab: 248,320
• —Context: 262,144 tokens

Usage with MTPLX

mtplx start --model wang-yang/Agents-A1-MTPLX-Q4

Usage with mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("wang-yang/Agents-A1-MTPLX-Q4")
prompt = "<|im_start|>user\nHello!<|im_end|>\n<|im_start|>assistant\n"
result = generate(model, tokenizer, prompt=prompt, max_tokens=200)

Notes

• —EOS token: <|im_end|> (id 248046)
• —MTP speculative decoding: ~1.33x speedup (D2 best, 101.8 tok/s vs AR 76.6 tok/s on M3 Max 128GB).

Files