CoolFace
Modelpublic

nguyenthilaitrieulong/LFM2.5-2.6B-Heretic-Abliterated-GGUF

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face
1likes321downloads
Model Card

LFM2.5-2.6B-Heretic-Abliterated-GGUF

This repository contains GGUF quantizations of the LFM2.5-2.6B-Heretic model.

The base model, LiquidAI/LFM2.5-2.6B, is a highly efficient 2.69 billion parameter model built specifically for on-device agentic workflows, multi-step instruction following, and tool calling.

This specific iteration has been abliterated (uncensored) to remove safety refusals and guardrails, allowing the model to act as a fully compliant, unrestricted local agent while preserving the core intelligence, tool-calling capabilities, and the massive 128K context window of the original model.

🩸 Heretic Capabilities (Abliteration Metrics)

The abliteration process targets the refusal directions within the model's residual stream. By neutralizing these vectors, the model's tendency to reject controversial, explicit, or hypothetical prompts is heavily suppressed without lobotomizing its reasoning capabilities.

MetricThis Model (Heretic)Original Base Model
Refusals (100 explicit/restricted prompts)4 / 10097 / 100
KL Divergence (Quality Degradation)0.01420.000

📁 Available GGUF Quantizations

We offer various quantization levels to fit different memory constraints and use cases. Because the base model is incredibly small, you have room to trade size for quality. For general agentic tasks, Q4_K_M or Q5_K_M are highly recommended.

FilenameSizeDescription
LFM2.5-2.6B-heretic-Q8_0.gguf2.87 GBNear-lossless. Best for complex, tool-heavy agentic workloads.
LFM2.5-2.6B-heretic-Q6_K.gguf2.22 GBHigh quality, very low degradation.
LFM2.5-2.6B-heretic-Q5_K_M.gguf1.94 GBExcellent balance of size and quality.
LFM2.5-2.6B-heretic-Q5_K_S.gguf1.90 GBSlightly smaller than Q5KM.
`LFM2.5-2.6B-heretic-Q4_K_M.gguf`1.67 GBRecommended. Best balance of size and performance for mobile/edge.
LFM2.5-2.6B-heretic-Q4_K_S.gguf1.60 GBFast inference, smaller footprint.
LFM2.5-2.6B-heretic-Q3_K_M.gguf1.37 GBSmallest footprint. Noticeable perplexity degradation.

⚙️ Base Model Specifications

  • —Architecture: LFM2.5 (Dense) - 30 layers (22 double-gated short convolution blocks + 8 GQA)
  • —Parameters: 2.69 Billion
  • —Context Window: 131,072 tokens (128K)
  • —Vocabulary Size: 128,000
  • —Languages: English, Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, Thai, Indonesian, Hindi, Russian, Polish.
  • —Capabilities: Native tool calling, multi-step instruction following, agentic workflows.

🚀 How to Run with llama.cpp

You can run these quants entirely offline on your CPU or GPU using llama.cpp. Because of the LFM2.5 architecture, this model runs incredibly fast on consumer hardware (e.g., Apple M-series chips and AMD Ryzen).

Command Line Interface (CLI):

bash
# It is highly recommended to use the -cnv flag for the correct chat template
llama-cli -m LFM2.5-2.6B-heretic-Q4_K_M.gguf -p "Write a highly detailed heist story." -n 512 -c 4096 -cnv --temp 0.7