datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
CausalReasoningBenchmark
Automated Causal Reasoning Benchmark
Overview
The Automated Causal Reasoning Benchmark is a collection of real-world causal inference tasks drawn from 73 peer-reviewed research papers and three textbook-style collections (see CausalBenchmark.pdf). The benchmark contains 174 queries over 139 datasets. Each task is designed to evaluate both (i) identification, i.e., selecting an appropriate causal estimand and identification strategy given the study context, and… See the full description on the dataset page: https://huggingface.co/datasets/syrgkanislab/CausalReasoningBenchmark.CausalReasoningBenchmark
Automated Causal Reasoning Benchmark
Anonymized benchmark release. Author, affiliation, and prior-whitepaper material have been removed.
Overview
The Automated Causal Reasoning Benchmark is a collection of real-world causal inference tasks drawn from 73 peer-reviewed research papers and three textbook-style collections. The benchmark contains 174 queries over 139 datasets. Each task is designed to evaluate both (i) identification, i.e., selecting an appropriate… See the full description on the dataset page: https://huggingface.co/datasets/anonsubmission16/CausalReasoningBenchmark.causal_factors
Causal Factors Dataset
This dataset provides 'causal factors' for each sample in MMLU, BIG-Bench Hard (BBH) (specifically, the thirteen problem subset used in Turpin et al., 2023), and GPQA (specifically, GPQA Diamond). We use this dataset to test for the faithfulness and verbosity of models' chain of thought reasoning, to better understand how we can measure monitorability.
For each problem in these constituent datasets, we use a panel of judge models to extract any factors that… See the full description on the dataset page: https://huggingface.co/datasets/ameek/causal_factors.