KRLabsOrg/LFM2.5-Encoder-350M-hallucination-detector

LFM2.5-Encoder-350M hallucination detector

<p align="center"> <img src="https://huggingface.co/KRLabsOrg/LFM2.5-Encoder-350M-hallucination-detector/resolve/main/mascot.png" alt="LettuceDetect mascot" width="360"/> </p>

A span-level hallucination detector built on LiquidAI/LFM2.5-Encoder-350M: the bidirectional LFM2.5 backbone with a linear token-classification head, fine-tuned to mark the character spans of an answer that are not supported by the given context. Part of the LettuceDetect project.

Trained on the LettuceDetect unified benchmark: coding-agent answers (SWE-bench-derived), developer tool output, structured documents (ACL papers, READMEs, Wikipedia markdown), RAGTruth, and 14-language PsiloQA.

Results

Character-level span metrics on the LettuceDetect unified test split (n=10,698):

Comparison

Span-F1 by source against the other LettuceDetect detectors on the same test split:

On multilingual example-level detection it matches the mmBERT encoder (PsiloQA example-F1 0.945 vs 0.943). On code-agent answers it remains far above general-purpose LLM judges at a fraction of their size (Nemotron-3-Ultra-550B 0.216, gpt-oss-120b 0.212 span-F1; HHEM-2.1 / Lynx-8B / Granite-Guardian / MiniCheck ≈ chance).

Usage

# pip install lettucedetect
from lettucedetect.models.inference import HallucinationDetector

detector = HallucinationDetector(
    method="transformer",
    model_path="KRLabsOrg/LFM2.5-Encoder-350M-hallucination-detector",
    trust_remote_code=True,
)

predictions = detector.predict(
    context=["The Eiffel Tower is 330 metres tall and stands in Paris, France."],
    question="How tall is the Eiffel Tower and where is it?",
    answer="The Eiffel Tower is 330 metres tall and stands in Berlin.",
    output_format="spans",
)
print(predictions)
# [{'start': 49, 'end': 56, 'confidence': 0.92, 'text': ' Berlin'}]

Token-level classification without the LettuceDetect wrapper:

import torch
from transformers import AutoTokenizer, AutoModelForTokenClassification

repo = "KRLabsOrg/LFM2.5-Encoder-350M-hallucination-detector"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForTokenClassification.from_pretrained(repo, trust_remote_code=True).eval()

enc = tokenizer("context text ... answer text", return_tensors="pt")
with torch.no_grad():
    labels = model(**enc).logits.argmax(-1)[0]   # 0 = supported, 1 = hallucinated

Training

• —Backbone: LiquidAI/LFM2.5-Encoder-350M (bidirectional), linear head, dropout 0.1
• —3 epochs on the unified train split (66,368 samples), input [question, context, answer], answer tokens labeled supported/hallucinated, max length 8,192
• —Token-level validation F1 0.635, test F1 0.609

Citation

@misc{kovács2026documentgroundingspanlevelhallucination,
      title={Beyond Document Grounding: Span-Level Hallucination Detection over Code, Tool Output, and Documents},
      author={Ádám Kovács and Bowei He and Xue Liu and István Boros and Szilveszter Tóth and Gábor Recski},
      year={2026},
      eprint={2607.00895},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2607.00895},
}