CoolFace
Modelpublic

xunkutech-ai/Qwythos-9B-Claude-Mythos-5-1M-MLX-oQ6-mtp

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face
1likes91downloads
Model Card

Qwythos-9B-Claude-Mythos-5-1M-MLX-oQ6-mtp

An oQ6 quantized MLX version of Qwythos-9B-Claude-Mythos-5-1M-MLX-bf16-mtp with integrated Multi-Token Prediction (MTP) head for accelerated speculative decoding on Apple Silicon.

Quantized with oMLX oQ6 — a higher-precision mixed-quantization variant that preserves more layers at higher fidelity for better quality at the cost of increased memory usage compared to oQ4.

Model Summary

  • —Format: MLX safetensors, oQ6 quantized
  • —Precision: Mixed (oQ6 smart quantization)
  • —Parameters: ~9B total
  • —Context length: 1,048,576 tokens
  • —Architecture: Qwen3.5-style hybrid attention text model with MTP head
  • —MTP layers: 1 (embedded, mtp_num_hidden_layers: 1)
  • —Primary use: local text generation and reasoning on Apple Silicon with MTP-accelerated decoding
  • —Base model: xunkutech-ai/Qwythos-9B-Claude-Mythos-5-1M-MLX-bf16-mtp

Compatibility Notice

[oMLX](https://github.com/jundot/omlx) is required to load this model with MTP support. Standard mlx-vlm, mlx-lm, and LM Studio do NOT support MTP and will fail with: ValueError: Received 15 parameters not in model: language_model.mtp.*.

Installation

bash
pip install -U git+https://github.com/jundot/omlx.git

Usage with oMLX

bash
# Start server with MTP draft-mode
omlx serve --model /path/to/Qwythos-9B-Claude-Mythos-5-1M-MLX-oQ6-mtp
python
from omlx import load, generate

model_id = "/path/to/Qwythos-9B-Claude-Mythos-5-1M-MLX-oQ6-mtp"
model, tokenizer = load(model_id)

result = generate(
    model=model,
    tokenizer=tokenizer,
    prompt="Explain speculative decoding with MTP.",
    max_tokens=2048,
    temperature=0.6,
)
print(result)

Files

FileSizeDescription
config.json~12 KBoQ6 quantized config with mtp_num_hidden_layers: 1
model-00001-of-00002.safetensors~5.0 GBQuantized shard 1/2
model-00002-of-00002.safetensors~3.5 GBQuantized shard 2/2 (+ MTP head)
tokenizer.json~20 MBQwen3.5 tokenizer
chat_template.jinja~8 KBQwythos chat template

Recommended Sampling

python
generation_kwargs = {
    "temperature": 0.6,
    "top_p": 0.95,
    "top_k": 20,
    "repetition_penalty": 1.05,
    "max_tokens": 4096,
}

License

Released under the same license as the upstream model: Apache-2.0.

Acknowledgements