xlr8harder/counterfactual-trace-audits
Counterfactual Trace Audits This dataset contains 25,600 unique synthetic, self-contained reasoning problems. Each problem shows an original computation over a list or binary tree, applies a counterfactual semantic patch, and asks for two K/R/X judgments plus both complete patched evaluation traces. Prompt format v2 explicitly defines trace notation and the nested answer schema. Tree-height prompts also include a small example of the pruning marker. The displayed answer shape… See the full description on the dataset page: https://huggingface.co/datasets/xlr8harder/counterfactual-trace-audits.
Add Qwen3 30B-A3B v2 baseline and resumable evaluator
Remove candidate shortcut and add v2 trace baseline
Add verifier and Flash smoke baseline
Clarify prompt and model projection contract
Declare MIT license
Publish Counterfactual Trace Audits v1
initial commit