mars2titan/amiko-hermit-1.0-27b-vl-mlx-4bit

Amiko Hermit 1.0

27B vision-language model for tool use. Fine-tuned from Qwen3.8-27B on OpenHermit traces, then converted to 4-bit MLX for Apple Silicon. Vision encoder and MTP draft head are kept.

Eval

Held-out tool-call split, 260 calls, greedy decode, 256 new tokens. Compared to the untuned Qwen3.8-27B base.

Training

Loss 3.18 → 0.62, token accuracy 47% → 85% over 250 steps (1 epoch). Curves are on the TensorBoard tab.

Use with MLX

pip install -U mlx-vlm

python -m mlx_vlm.generate \
  --model mars2titan/amiko-hermit-1.0-27b-vl-mlx-4bit \
  --max-tokens 256 --temperature 0.0 \
  --prompt "Describe this image." --image <path>

python -m mlx_vlm.server --model mars2titan/amiko-hermit-1.0-27b-vl-mlx-4bit

Use mlx-vlm, not mlx-lm, so vision stays.

Specs

• —Architecture: Qwen3.5 vision-language (Qwen3_5ForConditionalGeneration)
• —Quant: affine 4-bit, group size 64, 4.695 bits/weight, ~15 GB (3 shards)
• —Train: 1 epoch, 250 steps, QLoRA r=16 α=32. Loss 3.18 → 0.62, token accuracy 47% → 85%
• —License: Apache 2.0