CoolFace
Modelpublic

popfido/Qwythos-9B-Claude-Mythos-5-1M-oQ4-MLX-MTP-Visual

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face
0likes66downloads
Model Card

Qwythos-9B-Claude-Mythos-5-1M — oQ4 MLX (MTP + Vision)

4-bit oMLX-quantized MLX conversion of [empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF](https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF) (Qwen3.5-9B, hybrid GatedDeltaNet + attention) with the MTP head and the Qwen3.5-9B vision tower.

Pipeline: MTP-BF16 GGUF → float16 MLX (via [popfido/gguf2mlx](https://github.com/popfido/gguf2mlx/tree/feat/qwen35-mtp-vision)) → oQ level 4 quantization (oMLX Universal Dynamic Quantization). ~5.8 GB, runs on much less RAM than the full-precision variant. For maximum fidelity use the BF16 repo; for the un-quantized 8-bit source see Q8.

Quantization

  • —oQ4 — 4-bit base, mixed precision: sensitivity-driven higher bits (5–6) on the GatedDeltaNet in_proj_a/b, out_proj, and down_proj layers; affine, group size 64.
  • —MTP preserved (preserve_mtp=True): the mtp.* tensors and mtp_num_hidden_layers are kept so the Native MTP toggle works after quantization.
  • —Minor quality drift vs full precision is expected at 4-bit (e.g. occasional factual slips on greedy decode); core generation is coherent and correct.

The underlying float16 conversion was verified bit-exact vs llama.cpp (text), and the vision tower matches llama.cpp mtmd output — see the BF16 repo card for the full verification + Qwen3.5 conversion notes (zero-centered RMSNorm, A_log = log(−ssm_a), strided→grouped value-heads, YaRN).

Usage

python
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("popfido/Qwythos-9B-Claude-Mythos-5-1M-oQ4-MLX-MTP-Visual")
prompt = apply_chat_template(processor, model.config, "Describe this image.", num_images=1)
print(generate(model, processor, prompt, image=["image.png"], max_tokens=128).text)

Recommended sampling: temperature 0.6, top_p 0.95, top_k 20, repetition_penalty 1.05.

License & attribution

Inherits apache-2.0 from the base model. All credit for the model to empero-ai; this repo is an MLX-format conversion + oQ quantization only.