hermitdave/Agnes-3.0-Flash-MLX-6bit-MTP

Agnes-3.0-Flash — MLX 6-bit + MTP

6-bit MLX quantization of Agnes-AI/Agnes-3.0-Flash (Apache-2.0), grafted with Qwen3.8's vision tower and Agnes's native MTP head for enhanced inference.

• —Affine 6-bit, group size 64 — 6.50 bits/weight, ~21.5GB (5 shards)
• —262,144-token context, thinking on/off via the original chat template
• —MTP head grafted: Agnes native MTP head with FFN padded 17408→19456 (zero-pad, mathematically identical), quantized to 4-bit
• —Vision tower grafted from Qwen3.8 (333 bf16 tensors, byte-identical to Agnes original)

Architecture

Agnes-3.0-Flash is a 33B hybrid attention model:

• —54 recurrent (Gated DeltaNet) layers + 18 global attention layers
• —GQA 6:1 (24q/4kv heads, dim 256), 3-axis RoPE, causal conv kernel 4
• —Hidden size 5120, intermediate size 17408 (Agnes native)
• —MTP head intermediate size 19456 (Qwen3.8 compatible, padded from 17408)

What changed

Converted from the original Agnes format to standard Qwen3.5 architecture:

• —Folded parallel FFN into main MLP via concatenation (intermediate_size: 19456)
• —Renamed delta_attn → linear_attn, global_attn → self_attn
• —Converted one-centered RMSNorm to standard format
• —Cast bf16 → fp16 for serialization compatibility

MTP graft:

• —Padded Agnes native MTP head FFN layers (gateproj, upproj: +2048 rows; down_proj: +2048 cols) to match Qwen3.8's expected 19456
• —Quantized MTP weights to 4-bit affine (group_size=64)
• —Grafted 333 vision tensors from Qwen3.8 (byte-identical to Agnes original)

Benchmark

Measured on M3 Max 64GB, oMLX, temperature 0:

Usage

pip install mlx-lm
mlx_lm.generate --model hermitdave/Agnes-3.0-Flash-MLX-6bit-MTP --prompt "Hello" --max-tokens 200

Or with oMLX: place under ~/.omlx/models/hermitdave/ and enable mtp_enabled + vlm_mtp_enabled in model settings.

Attribution

This conversion was produced by Hermes Agent (Nous Research). MTP graft methodology: pad Agnes native MTP head FFN to Qwen3.8 size (19456), quantize to 4-bit, graft Qwen3.8 vision tower.