vcruz305/K2-Horizon-MoVA-36B-A4B-EXL3

K2-Horizon-MoVA-36B-A4B · EXL3

EXL3 quantizations of IFM's K2-Horizon-MoVA-36B-A4B, one folder per bitrate. Every pack is scored against the BF16 original on held-out text before it is uploaded.

• —top-1: positions where the pack's most likely next token matches the reference's, counted over 10,240 positions of held-out text.
• —KLD: KL(reference ‖ pack) per position over the same positions, mean and 99th percentile. Lower is closer.
• —Reference: IFM's own implementation (modeling_k2_horizon.py) running the original BF16 weights with fp32 activations.
• —For scale: the unquantized BF16 weights running in the same exllamav3 code match the reference on 97.71% of positions (10,006 / 10,240) with mean KLD 0.0030. That is about the best any pack here can score.
• —size is the whole folder on disk. smallest card is the smallest GPU that holds it resident with room for KV cache. The 8.00 pack is 38 GB, so it runs on a 48 GB card as well as a 96 GB one.
• —The 2.00 row is a floor pack: about 11 GB of weights, so an 8 GB card needs expert offload.

About these quants

• —Made with SAGE, our own mixed-precision quantization method for EXL3. The number in each folder name is the average bitrate of the model body. The output head is 6-bit in every pack.
• —The scores compare each pack with the reference above on text that was not used to make the quants. They measure how closely a pack tracks BF16, not task accuracy.

What K2-Horizon-MoVA is

K2-Horizon-MoVA-36B-A4B is the sparse member of IFM's K2-Horizon family. It stores 36B parameters but runs about 4B per token, using two kinds of routing: a Mixture-of-Experts feed-forward block and Mixture-of-Values attention (MoVA), where each token's value projection is itself routed through a bank of experts. It is Apache-2.0, and IFM is publishing the training data, logs, intermediate checkpoints and code alongside the weights.

Highlights from IFM's model card:

• —Frontier-class results at 4B active parameters. IFM reports that on agentic and reasoning benchmarks it outscores open dense models of about 30B and MoE models up to 15× its size.
• —512K context. Native 524,288-token context from midtraining onward.
• —Intermediate checkpoints. Every training stage is published as a branch of the base repo (pretrain_*, mid_1_* … mid_4_*, sft_1_*, sft_2_*), so you can study how capability changes across training.
• —Fully open. Pretraining and midtraining data (IFM/K2-Horizon-Pretrain-Data, IFM/K2-Horizon-Midtrain-Data), training logs and a blog post are out. The code repository and technical report are listed for end of September 2026.

Architecture

From the model's config.json:

Training

IFM trained it in stages, each continuing from the previous one (about 25.1T tokens in total):

Reported results

Scores for the BF16 model from IFM's card (%, high reasoning effort, Artificial Analysis categories). They were not re-run on these quants. Use the top-1/KLD columns above to see how closely each pack tracks BF16.

Using it well

IFM's recommended settings, which apply to these packs as well:

• —Reasoning effort `high` on every request: chat_template_kwargs: {"reasoning_effort": "high"}. All of IFM's reported results use it.
• —Sampling: temperature=1.0, top_p=0.95.
• —Thinking comes back in reasoning_content and the answer in content when your server uses a K2-Horizon reasoning parser.
• —Tool calls can use json, xml (default) or xml_typed, selected with chat_template_kwargs: {"tool_call_format": ...}.
• —Each folder ships the original chat_template.jinja and tokenizer.

Running

These packs load in exllamav3 with the K2-Horizon architecture port (K2HorizonForCausalLM: MoVA values, softplus attention gate, grouped RMSNorm, selection-only router bias). Upstream exllamav3 does not include this architecture yet. The port and a serving recipe with RTX PRO 6000 speed numbers will be linked here when they are published.

License and credit

Apache-2.0, same as the base model. The model, data and training are IFM's work. See the original model card for full details.

@misc{k2horizon2026,
  title  = {Introducing K2 Horizon: Frontier Performance, Radically Open},
  author = {{IFM Team}},
  year   = {2026},
  url    = {https://ifm.ai/blog/k2/},
}