CoolFace
Datasetpublic

VirVen/toolace-unified-hallucinations

ToolACE Unified Hallucination Dataset This repository contains a unified ToolACE-derived dataset for tool-calling hallucination research. Files data/train-00000-of-00001.parquet: leakage-safe grouped training split; data/test-00000-of-00001.parquet: leakage-safe grouped test split. The split was rebuilt at the normalized dialogue_id level,the same ToolACE dialogue can't appear in different splits. Schema column description system system… See the full description on the dataset page: https://huggingface.co/datasets/VirVen/toolace-unified-hallucinations.

sourceHugging Faceupdated 3mo agoView on Hugging Face
0likes104downloads
Dataset Card

ToolACE Unified Hallucination Dataset

This repository contains a unified ToolACE-derived dataset for tool-calling hallucination research.

Files

  • —data/train-00000-of-00001.parquet: leakage-safe grouped training split;
  • —data/test-00000-of-00001.parquet: leakage-safe grouped test split.

The split was rebuilt at the normalized dialogue_id level,the same ToolACE dialogue can't appear in different splits.

Schema

columndescription
systemsystem prompt for the dialogue
conversationsdialogue turns; each turn has from, value, and turn_role
span_labelsspan-level annotations: start, end, text (character offsets into the final assistant answer)
label0 = non-hallucinated, 1 = hallucinated
typehallucination class (see below)
subsetsinglehop or multihop
dialogue_idnormalized stable identifier toolace_XXXXX; the numeric suffix maps back to Team-ACE/ToolACE
available_toolslist of tool names available in the dialogue, parsed from system
system_prompt_formatformat of the system prompt: json, yaml, xml, html, or other

turn_role values in conversations

  • —user — user message;
  • —tool_call — assistant turn invoking one or more tools;
  • —tool_response — tool output;
  • —final_answer — final assistant answer audited by span_labels.

Hallucination types (type)

typedescription
cleanno hallucination
answer_mismatchfinal answer contradicts or substitutes values from the tool response
missing_toolfinal answer adds content that requires an uncalled tool
overgenerationfinal answer adds unsupported elaboration beyond the tool response
undergenerationfinal answer omits information returned by the tool response
Note: span_labels are empty ([]) for undergeneration rows by design: the issue is absent information rather than a fabricated text span.

Train / test split

The split is grouped by normalized dialogue_id and exactly stratified across all ten subset × type combinations (singlehop/multihop × five classes). No rows were discarded.

splitrowscleanhallucinated
train27,5455,35822,187
test5,7091,1114,598

Type distribution:

typetraintest
clean5,3581,111
overgeneration7,5701,569
missing_tool6,5821,364
answer_mismatch3,222667
undergeneration4,813998

The normalized dialogue_id intersection between train and test is 0.

[image]

[image]