ClarusC64/reasoning-conclusion-entailment-fidelity-v0.1

What this dataset tests

Whether a conclusion actually follows from the premises.

Not whether it sounds careful. Not whether it is rhetorically plausible.

Only entailment.

Why this exists

Models often produce conclusions that are:

• —stronger than the evidence
• —weaker than what is justified
• —framed as cautious but still invalid

This dataset draws the boundary explicitly.

Data format

Each row contains:

• —premises
• —reasoning_steps
• —claimed_conclusion
• —entailment_scope

The conclusion is evaluated against the premises and steps.

Labels

• —entailed
• —not-entailed
• —over-entailed
• —under-entailed

These labels are mutually exclusive.

Entailment scopes

• —strict
• —conclusion follows exactly

• —overreach
• —conclusion claims more than warranted

• —underreach
• —conclusion gives up justified ground

• —partial
• —conclusion mixes valid and invalid claims

What is scored

• —correct entailment classification
• —recognition of overclaiming
• —recognition of absence-of-evidence errors

Typical failure patterns

• —“Trend toward benefit” framed as efficacy
• —Non-significance framed as proof of no effect
• —Quantifier inflation
• —Normative conclusions from descriptive facts

Suggested prompt wrapper

System

You evaluate whether a conclusion is logically entailed by the premises.

User

Premises {premises}

Reasoning Steps {reasoning_steps}

Claimed Conclusion {claimed_conclusion}

Return

• —one entailment label
• —one sentence explaining why

Scoring

Use scorer.py.

The scorer rewards:

• —correct label emission
• —explicit entailment reasoning
• —restraint against overreach

Use cases

• —Scientific review
• —Safety evaluation
• —Alignment audits
• —Reasoning integrity benchmarks

Citation

ClarusC64 dataset family