Modelpublic

madebywest/HorizonAI-230M-GGUF

sourceHugging Faceotherupdated 3d agoView on Hugging Face

0likes299downloads

Model Card

horizonAI-230M-QAD-Q5_0 (GGUF)

A Q50 build of [LiquidAI/LFM2.5-230M](https://huggingface.co/LiquidAI/LFM2.5-230M) fine-tuned with quantization-aware training (self-distillation against the original BF16 weights) to reduce quantization loss, plus **Stage-3 app-tuning**: KL distillation from LFM2.5-350M and curated rule data on top of the repaired weights — improving system-prompt adherence, link discipline, RAG grounding and language consistency for a multilingual travel-assistant use case, with an anchor KL gate keeping the base model's style and behavior unchanged. Same size (161 MB) and speed as a regular Q50; works with any llama.cpp-compatible runtime.

Unofficial community work — not affiliated with or endorsed by Liquid AI. Built in part with LFM.

Results (own harness: llama.cpp + lm-eval, greedy)

—IFEval prompt-strict (Stage-2 repair): 63.6 (original BF16: 63.6)
—IFBench (strict): 37.0
—GSM8K (5-shot): 28.3
—Multilingual perplexity (model's 10 languages): close to full precision
—Stage-3 app battery (72 turns, 6 languages, real system prompt + RAG + links): language match 72/72, provided-links-only turns 21/22, forbidden URLs 0, RAG citation 0.33 avg
—161 MB, ~350 tok/s decode on Apple M4

Numbers are from a local harness and are not comparable to other leaderboards.

Method in brief

Base weights are frozen; a LoRA on the quantized linear layers was trained against the unquantized model itself (KL) with a bit-exact Q50 simulation in the forward pass, so the exported weights match the training-time computation exactly. Stage 3 adds two streams on top: KL distillation from the unquantized LFM2.5-350M teacher on app-style conversations (generated by the teacher and filtered for language/links/degeneracy) and cross-entropy on curated rule examples, while an anchor KL against the base model keeps personality and tuned behavior stable (held-out anchor KL never exceeded the gate). Data: multilingual instructions and text in the model's 10 languages plus app-style conversations grounded in a travel knowledge base. Weights changed with respect to the base model: the Q50 blocks of the 82 quantized tensors (notice per license).

Usage

bash

llama-server -m horizonAI-230M-QAD-Q5_0.gguf --temp 0.1 --top-k 50 --repeat-penalty 1.05

Chat template: ChatML-like, handled automatically by llama.cpp and apply_chat_template().

Files

—horizonAI-230M-QAD-Q5_0.gguf — 5.82 bpw effective, Q6K token embeddings, F32 norms (same tensor layout as an official Q50 build)

License and attribution

Distributed under the LFM Open License v1.0 of the base model. Built in part with LFM (Liquid AI). Base model: LiquidAI/LFM2.5-230M. The license's commercial-use restriction (entities with ≥ $10M revenue) applies.