CoolFace
Modelpublic

katsubakirill/toolaware-modernbert-large-toolace-hallucination

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face
0likes9downloads
Model Card

ToolAwareModernBERTlarge_finetuned

Fine-tuned ModernBERT token classifier for span-level hallucination detection in tool-calling dialogues.

Base Model

KRLabsOrg/lettucedect-large-modernbert-en-v1

Dataset

katsubakirill/toolace-hallucination-span-detection

Input Format

tool-aware input: question, available tools, executed tool call, tool facts, raw tool output, answer

Only answer tokens are trained/evaluated for hallucination labels. Non-answer tokens are ignored with label -100.

Labels

  • —0: non_hallucinated
  • —1: hallucinated

Test Metrics

dataset_typen_examplesspan_precisionspan_recallspan_f1example_f1exact_span_match_ratethreshold
overall3300.8571430.7730280.8129150.8039870.5090910.99
contradictionwithtool_output1300.3861850.3125790.3455060.6078430.4769230.99
missing_tool620.9102040.8889520.8994530.9206350.6451610.99
overgeneration1380.9123320.8102720.8582780.8970590.4782610.99