Modelpublic

ZQ-Dev/KAT-Coder-V2.5-Dev-oQ4e-fp16

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

2likes219downloads

Model Card

KAT-Coder-V2.5-Dev-oQ4e

A 4-bit enhanced oQ quantization of Kwaipilot/KAT-Coder-V2.5-Dev for Apple Silicon and MLX-compatible runtimes.

Quantization details

The full calibration summary is available in `oq_imatrix_report.json`.

Note: This quant was created using float16 non-quant weights. Per oMLX, float16 gives ~20% faster prefill on M1/M2 Apple Silicon (native fp16). bfloat16 is safer on M3+ and for numerical stability. For the standard oQe with bfloat16, see this repo.

License

Apache 2.0, inherited from the base model.