CoolFace
Modelpublic

xunkutech-ai/Qwythos-9B-Claude-Mythos-5-1M-MLX-bf16-mtp-draft

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face
0likes57downloads
Model Card

Qwythos-9B Claude Mythos 5 1M - MTP Draft Model (MLX)

This is a Qwen3.5 Multi-Token Prediction (MTP) draft model extracted from the Qwythos-9B Claude Mythos 5 1M context model, converted to MLX format for use with mlx-vlm speculative decoding.

Model Details

PropertyValue
Base Modelempero-ai/Qwythos-9B-Claude-Mythos-5-1M
Source GGUFQwythos-9B-Claude-Mythos-5-1M-MTP-BF16.gguf (17GB)
MLX Formatbfloat16 / float32
ArchitectureQwen3.5 MTP Draft Model
Context Length1,048,576 tokens (1M)
Layers32 (main) + 1 MTP layer
Hidden Size4096
Intermediate Size12288
Attention Heads16
KV Heads4
Head Dim256
RoPEYARN, partialrotaryfactor=0.25, factor=4.0
Block Size3
MTP Num Hidden Layers1
Weight Count15 tensors
Weight Size~464 MB

What is MTP?

Multi-Token Prediction (MTP) is a speculative decoding technique where a smaller "draft" model predicts multiple tokens ahead, which are then verified by the main model in a single pass. This can significantly speed up inference without sacrificing quality.

This draft model extracts the MTP projection weights (blk.32.nextn.*) and the MTP attention/FFN blocks from the original model, producing a standalone qwen3_5_mtp compatible drafter.

Usage

With mlx_vlm CLI

bash
mlx_vlm.generate \
  --model /path/to/Qwythos-9B-Claude-Mythos-5-1M-MLX-bf16 \
  --draft-model /path/to/Qwythos-9B-Claude-Mythos-5-1M-MLX-bf16-mtp-draft \
  --draft-kind mtp

With mlx_vlm Python API

python
from mlx_vlm import load_model, generate
from mlx_vlm.utils import load_prompt_cache

model = load_model("/path/to/Qwythos-9B-Claude-Mythos-5-1M-MLX-bf16")
draft_model = load_model("/path/to/Qwythos-9B-Claude-Mythos-5-1M-MLX-bf16-mtp-draft")

# Use draft_model with --draft-kind mtp equivalent

Weights

KeyShapeDtype
fc.weight(4096, 8192)bfloat16
layers.0.input_layernorm.weight(4096,)float32
layers.0.post_attention_layernorm.weight(4096,)float32
layers.0.self_attn.q_proj.weight(8192, 4096)bfloat16
layers.0.self_attn.k_proj.weight(1024, 4096)bfloat16
layers.0.self_attn.v_proj.weight(1024, 4096)bfloat16
layers.0.self_attn.o_proj.weight(4096, 4096)bfloat16
layers.0.self_attn.q_norm.weight(256,)float32
layers.0.self_attn.k_norm.weight(256,)float32
layers.0.mlp.gate_proj.weight(12288, 4096)bfloat16
layers.0.mlp.up_proj.weight(12288, 4096)bfloat16
layers.0.mlp.down_proj.weight(4096, 12288)bfloat16
norm.weight(4096,)float32
pre_fc_norm_embedding.weight(4096,)float32
pre_fc_norm_hidden.weight(4096,)float32

Extraction Process

  1. 1.Source GGUF (Qwythos-9B-Claude-Mythos-5-1M-MTP-BF16.gguf) parsed for MTP tensors at blk.32
  2. 2.15 MTP tensors extracted (BF16/FP32 only, lossless)
  3. 3.Keys mapped to mtp.* format for mlx-vlm splitter
  4. 4.Official mlx_vlm.speculative.drafters.qwen3_5_mtp.split applied to produce final drafter
  5. 5.Verified with strict=True load against Qwen3_5MTPDraftModel

License

Apache 2.0 (inherited from base model)

Related