Modelpublic

FancieF/Qwythos-27B-v1-MLX-VLM-bf16

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

0likes87downloads

Model Card

Qwythos-27B-v1 · MLX VLM BF16

Unquantized bfloat16 MLX conversion of empero-ai/Qwythos-27B-v1 for Apple Silicon. Vision tower kept.

Not the everyday download. Prefer 4-bit VLM on ≤32–48GB Macs. This build is ~55GB and needs high unified memory.

Which repo should I download?

Repo	Size	Use when
VLM 4-bit	~15GB	Default chat / vision
This BF16	~55GB	Max fidelity, further quant, Studio-class RAM
MTP drafter	~829MB	Optional `--draft-model` speed-up

Install

bash

pip install -U mlx-vlm

Quick start

bash

mlx_vlm.generate \
  --model FancieF/Qwythos-27B-v1-MLX-VLM-bf16 \
  --prompt "Introduce yourself in one sentence." \
  --thinking-mode disabled \
  --max-tokens 256 \
  --temperature 0.6

Vision

bash

mlx_vlm.generate \
  --model FancieF/Qwythos-27B-v1-MLX-VLM-bf16 \
  --image ./photo.jpg \
  --prompt "Describe this image." \
  --thinking-mode disabled

+ MTP

bash

mlx_vlm.generate \
  --model FancieF/Qwythos-27B-v1-MLX-VLM-bf16 \
  --draft-model FancieF/Qwythos-27B-v1-MLX-MTP \
  --draft-kind mtp \
  --draft-block-size 4 \
  --prompt "..." \
  --thinking-mode disabled

LM Studio / memory

—BF16 27B + KV easily exceeds 32GB — expect swap or OOM on common MacBooks.
—For LM Studio with MTP, use official GGUF MTP quants instead of this MLX BF16.

What’s inside

—MLX safetensors, BF16, no weight quantization
—language_model + vision_tower
—No baked-in MTP (same as 4-bit; pair with the MTP repo)

Conversion

bash

mlx_vlm.convert \
  --hf-path /path/to/Qwythos-27B-v1 \
  --mlx-path ./Qwythos-27B-MLX-VLM-bf16 \
  --dtype bfloat16 --trust-remote-code

(No -q.)

License & attribution

Derived from empero-ai/Qwythos-27B-v1 (Apache-2.0), based on Qwen3.5-27B. Unofficial MLX conversion.

Links


Recommended 4-bit	FancieF/Qwythos-27B-v1-MLX-VLM-4bit
MTP	FancieF/Qwythos-27B-v1-MLX-MTP
Base	empero-ai/Qwythos-27B-v1
ModelScope (ZH)	Fancie/Qwythos-27B-v1-MLX-VLM-bf16