Modelpublic

wang-yang/Ornith-1.5-9B-MTPLX-4bit

sourceHugging Facemitupdated 29d agoView on Hugging Face

1likes782downloads

Model Card

Ornith-1.5-9B-MTPLX

A 4-bit MLX/MTPLX quantization of ornith-ai/Ornith-1.5-9B for Apple Silicon. The native one-layer MTP head is included as a quantized sidecar, along with the vision tower.

Quantization

—Main model: 4-bit affine, group size 64
—MTP sidecar: 4-bit affine, group size 64 (mtp.safetensors)
—MTP normalization tensors: BF16
—Vision tower: BF16 (model-vision.safetensors)

Usage

bash

mtplx start cli --model <path-to-this-model>
mtplx tune --model <path-to-this-model> --retune

The upstream model is licensed MIT. MTPLX: https://github.com/youssofal/MTPLX