RenShiPDev/LFM2.5-1.2B-Thinking-LeetCode-QLoRA-GGUF

LFM2.5-1.2B-Thinking-LeetCode-QLoRA — GGUF

Quantized GGUF builds of the LoRA adapter `RenShiPDev/LFM2.5-1.2B-Thinking-LeetCode-QLoRA`, merged into `LiquidAI/LFM2.5-1.2B-Thinking`.

QLoRA fine-tune (500 steps on greengerong/leetcode), converted for CPU / llama.cpp inference. Research checkpoint, not a production model.

All quants were produced from the BF16 merge with llama-quantize (no requantization).

Quality vs BF16

Measured on WikiText-2 (test split, ~246k tokens) with llama-perplexity --kl-divergence, teacher-forced. BF16 is the reference: lower PPL is better, and KLD mean is the mean KL divergence between this quant's token distribution and BF16 (0 = identical).

Recommendation: Q4_K_M — best size/quality trade-off (~697 MB, +0.38 PPL, KLD 0.048). Q6_K/Q8_0 are near-lossless. Below Q3_K_M degradation becomes large.

How it was measured

# 1. save reference logits (once)
llama-perplexity -m LFM2.5-1.2B-Thinking.BF16.gguf -f wiki.test.raw -c 512 -ngl 99 \
  --save-all-logits base_logits.bin

# 2. PPL + KL divergence per quant
llama-perplexity -m <quant>.gguf -f wiki.test.raw -c 512 -ngl 99 \
  --kl-divergence --kl-divergence-base base_logits.bin

Usage

# CLI
llama-cli -m LFM2.5-1.2B-Thinking.Q4_K_M.gguf -p "Explain binary search."

# Server (OpenAI-compatible)
llama-server -m LFM2.5-1.2B-Thinking.Q4_K_M.gguf --port 8080

Source

• —Adapter: RenShiPDev/LFM2.5-1.2B-Thinking-LeetCode-QLoRA
• —Base model: LiquidAI/LFM2.5-1.2B-Thinking
• —Training data: greengerong/leetcode