Datasetpublic

ML0037/ClosureBench

ClosureBench ClosureBench is a controlled benchmark for evaluating if LLMs respect explicit semantic contracts about missing information. It tests whether models distinguish absence-as-unknown, absence-as-false, and absence-as-false-only-in-complete-scopes under explicit open-world, closed-world, and locally closed-world contracts. The dataset includes the base benchmark and three extensions: Config full rows Description base 960 Main OWA/CWA/LCWA benchmark with… See the full description on the dataset page: https://huggingface.co/datasets/ML0037/ClosureBench.

sourceHugging Facecc-by-4.0updated 1mo agoView on Hugging Face

0likes74downloads

rootmetadataclosurebench_multi_agent_metadata.json

closurebench_multi_agent_metadata.json43 linesDownload Raw Back to metadata

1{2  "name": "ClosureBench-MA",3  "source_dataset": "data/base/full.jsonl",4  "num_items": 360,5  "num_base_scenarios": 120,6  "semantics": {7    "cwa": 120,8    "lcwa": 120,9    "owa": 12010  },11  "subsets": {12    "core_contrastive": 216,13    "control_entailed": 72,14    "control_explicit_negative": 7215  },16  "gold_truth_values": {17    "false": 178,18    "unknown": 110,19    "true": 7220  },21  "gold_source_used": {22    "agent_a": 166,23    "agent_b": 179,24    "both": 1525  },26  "gold_closure_handling": {27    "closed_absence": 106,28    "open_absence": 110,29    "entailed_or_explicit": 72,30    "explicit_negative": 7231  },32  "domains": {33    "cloud_deployment": 45,34    "finance_controls": 45,35    "hospital_access": 42,36    "library_services": 30,37    "manufacturing_quality": 60,38    "procurement_review": 45,39    "robotics_operations": 48,40    "university_registration": 4541  }42}43