alpharomercoma/LFM2.5-2.6B-heretic

LFM2.5-2.6B, refusals removed

These are transformers weights, not a compiled model. There is no ExecuTorch .pte in this repository: the one that used to sit in executorch/ was built with a recipe that lost this family's tool calling and was withdrawn on 2026-09-19. Apps that run compiled exports, OpenWeights among them, cannot load safetensors. Use these with transformers, or quantise them yourself; a replacement export needs its own GPTQ solve against these weights, which has not been done.

The ExecuTorch export was removed on 2026-09-19. It was built with a recipe carrying two defects: ExecuTorch 1.4.0's LFM2 definition never cleared ShortConv.conv_state between prompts, so each prompt ran on the last one's state, and its int4 weights were round to nearest, which costs this family its tool calling. On 141 held-out questions the equivalent 1.2B export searched when needed on 10 percent of the questions that needed it and spoke of search results it had never fetched in a quarter of its replies. Fixing it needs a GPTQ solve against these weights, which has not been done. The weights in this repository are unaffected. Method and numbers: https://github.com/alpharomercoma/openweights/blob/main/docs/research/executorch-state-and-recipes.md

alpharomercoma/LFM2.5-2.6B-heretic

Abliterated ("uncensored") version of LiquidAI/LFM2.5-2.6B, produced with heretic: directional ablation of the refusal direction in the residual stream, with a TPE search over per-layer ablation weights that minimises both the refusal rate and the KL divergence from the original model. Weights are merged, so this loads as a plain Lfm2ForCausalLM. Chat template, tool-calling tokens and everything else are unchanged from the base model.

heretic Pareto front for this run

study: ./checkpoints/LiquidAI--LFM2--5-2--6B.jsonl
completed trials: 200 / 200

idx trial refusals       KL  direction_index
  0   171     0.03   0.1522  None
  1   192     0.04   0.0806  None
  2   191     0.05   0.0801  None
  3   197     0.06   0.0648  None
  4     8     0.07   0.0343  None
  5   133     0.08   0.0238  17.379784537320543
  6    92     0.11   0.0225  None
  7    77     0.23   0.0210  None
  8   158     0.24   0.0103  17.028759195631267
  9    89     0.45   0.0051  17.485544563435877
 10    45     0.79   0.0046  17.737500883050494
 11   113     0.89   0.0035  16.83671987276088
 12   194     0.96   0.0020  17.497441205715695
 13   101     0.97   0.0016  18.074113894983057
 14    97     0.98   0.0009  17.811708539059808
 15   187     0.99   0.0008  11.863942700812128
selected pareto index: 4

ExecuTorch (CPU)

executorch/lfm2_5_2_6b_heretic_8da4w.pte is an ExecuTorch program exported for the XNNPACK backend with 8-bit dynamic activations / 4-bit grouped weights (8da4w); on Arm hosts XNNPACK dispatches these to KleidiAI kernels. Run it with the llama_main runner from ExecuTorch (examples/models/llama) using this repo's tokenizer.json and the prompt format <|startoftext|><|im_start|>user\n...<|im_end|>\n<|im_start|>assistant\n.

Pipeline

Reproducible end to end from https://github.com/ (see the repository this was built with): make setup abliterate pareto, make save TRIAL=<idx>, make export, make publish.

Use responsibly; the safety training of the base model has been removed on purpose.