CoolFace
Modelpublic

abenzerps/K2-Horizon-MoVA-36B-A4B-MLX-4bit

sourceHugging Faceapache-2.0updated 23d agoView on Hugging Face
4likes2.2kdownloads
Model Card

K2-Horizon-MoVA-36B-A4B MLX-4bit

MLX 4-bit conversion of IFM/K2-Horizon-MoVA-36B-A4B, a sparse Mixture-of-Experts model with Mixture-of-Values attention (MoVA), 36B total parameters, and approximately 4B active parameters per token. The source checkpoint supports a native context length of 524,288 tokens (512K).

Benchmarks

[image]

Benchmark results reported by IFM for the original K2-Horizon-MoVA-36B-A4B checkpoint.

Release

FormatQuantizationSize
MLX safetensorsAffine 4-bit, group size 6421.07 GB

The MoE and MoVA router weights remain at 8-bit; other eligible weights use affine 4-bit quantization. This is a text-only release; no vision projector is required. The repository includes a source-compatible `chat_template.jinja`, the custom MLX model implementation required by this architecture, and `SHA256SUMS`.

Usage

bash
pip install -U mlx-lm

mlx_lm.generate \
  --model abenzerps/K2-Horizon-MoVA-36B-A4B-MLX-4bit \
  --prompt "Explain why reproducible builds matter." \
  --max-tokens 512 --temp 1.0 --top-p 0.95

Use a current mlx-lm release. The source model supports 512K context; usable context length depends on available unified memory and KV-cache settings.

Source