CoolFace
Modelpublic

mlx-community/Qwythos-9B-v2-OptiQ-4bit

sourceHugging Faceapache-2.0updated 12d agoView on Hugging Face
3likes580downloads
5 commits on main
0967d0d12d ago

Update kv_config.json: re-measured with `optiq kv-cache` float32 sensitivity (the previous config was ranked on bfloat16 rounding noise). 8-bit layers [15] -> [7].

codelion
1e0bc1213d ago

Add kv_config.json: per-layer KV cache precision from `optiq kv-cache` (target 4.5 bits). Use with `optiq serve --kv-config kv_config.json`.

codelion
9c0cbe22mo ago

index: register the bf16 vision tower so stock VLM loaders (mlx-vlm, oMLX, LM Studio) can find it

codelion
17701593mo ago

OptiQ mixed-precision 4/8-bit quant (5.211 bpw) with bf16 vision sidecar + MTP head

codelion
15bad483mo ago

initial commit

codelion