CoolFace
Modelpublic

katsubakirill/toolaware-modernbert-base-toolace-hallucination

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face
0likes7downloads
Model Card

ToolAwareModernBERTfinetuned

Fine-tuned ModernBERT token classifier for span-level hallucination detection in tool-calling dialogues.

Base Model

KRLabsOrg/lettucedect-base-modernbert-en-v1

Dataset

katsubakirill/toolace-hallucination-span-detection

Input Format

tool-aware input: question, available tools, executed tool call, tool facts, raw tool output, answer

Only answer tokens are trained/evaluated for hallucination labels. Non-answer tokens are ignored with label -100.

Labels

  • —0: non_hallucinated
  • —1: hallucinated

Test Metrics

dataset_typen_examplesspan_precisionspan_recallspan_f1example_f1exact_span_match_ratethreshold
overall3300.7956510.7578960.7763150.806780.5060610.99
contradictionwithtool_output1300.2591820.313850.2839080.5714290.4692310.99
missing_tool620.9644470.8496580.9034210.950820.6290320.99
overgeneration1380.8584340.8020640.8292920.9117650.4855070.99