blaj/LFM2.5-2.6B-heretic-int4-ov

LFM2.5-2.6B-heretic int4 OpenVINO IR

An int4 OpenVINO IR conversion of heretic-org/LFM-2.5-2.6B-heretic, which is itself a decensored (abliterated) build of LiquidAI/LFM2.5-2.6B produced with Heretic v1.4.0.

This repository contains the converted OpenVINO IR only. No weight values were modified beyond int4 weight compression — the abliteration is inherited from the source model.

Conversion details

Files

openvino_model.xml / .bin          # main graph + int4 weights
openvino_tokenizer.xml / .bin      # tokenizer
openvino_detokenizer.xml / .bin    # detokenizer
config.json                        # model architecture config
generation_config.json
chat_template.jinja
tokenizer.json
openvino_config.json               # records export + quantization settings
LICENSE                            # LFM Open License v1.0

Usage

OpenVINO GenAI

import openvino_genai as og

pipe = og.LLMPipeline("lfm2.5-2.6b-heretic-int4-ov", "GPU")
config = og.GenerationConfig(max_new_tokens=256, temperature=0.7)

print(pipe.generate("User: Hello\nAssistant:", config))

OpenAI-compatible serving (OVMS)

ovms --configure --model_path ./lfm2.5-2.6b-heretic-int4-ov --task text_generation
ovms --model_path ./lfm2.5-2.6b-heretic-int4-ov --task text_generation --rest_port 11436

The model is then available at http://127.0.0.1:11436/v1/chat/completions.

Chosen format: int4

int4 is the smallest format published for this model. Abliteration removes the refusal direction from the weights, and aggressive quantization can erode other capabilities along with it, so an int8 sibling is published alongside this build for cases where the extra quality matters more than the size.

Verifying the conversion

The IR was tested by loading it through openvino_genai.LLMPipeline on an Intel Arc iGPU and running generation before publication. If you re-convert, do the same — a successful export does not guarantee a model that produces coherent output.

Reproducing

pip install optimum[openvino] nncf
pip install "transformers==5.4.0"   # hard requirement of the export gate

optimum-cli export openvino \
  --model heretic-org/LFM-2.5-2.6B-heretic \
  --task text-generation-with-past \
  --weight-format int4 \
  ./lfm2.5-2.6b-heretic-int4-ov

Benchmark

Single-stream, Intel Core Ultra 7 258V (Arc 130V/140V iGPU), 30 GB RAM, OpenVINO Model Server 2026.4.0 on GPU:

Full configuration, hardware detail and methodology: BENCHMARK.md.

Cross-model reference

Same harness, same hardware:

The 8B A1B is a mixture-of-experts model and decodes faster than the 2.6B dense model despite the larger checkpoint, because fewer parameters are active per token. Figures in this table are from one canonical run of benchmark_ov_models.py; expect a few percent variance between runs.

Attribution

• —Base model: LiquidAI/LFM2.5-2.6B (LFM Open License v1.0)
• —Abliteration: heretic-org/LFM-2.5-2.6B-heretic
• —Conversion: OpenVINO IR via optimum-intel

Redistribution is permitted under the LFM Open License v1.0, which is included verbatim in this repository. The license's commercial-use threshold applies to this Derivative Work as it does to the source.

Intended use

Research and local inference. Abliterated models have reduced refusal behavior; outputs should be evaluated accordingly and not deployed without appropriate consideration.