stephaniebrandl/climate_fever_rationales
The Climate-Fever dataset was first collected and published by Diggelmann et al, 2020.For our study, we are interested in token-level rationales which are not available from the initial publication of Climate-Fever. Therefore, we manually selected a subset of 102 claims (510 claim-evidence pairs) based on clarity of the claim formulation and balanced claim labels. Each sample was annotated on token-level by 3 annotators as either supporting the claim (label=1), contradicting the claim… See the full description on the dataset page: https://huggingface.co/datasets/stephaniebrandl/climate_fever_rationales.
The Climate-Fever dataset was first collected and published by Diggelmann et al, 2020. For our study, we are interested in token-level rationales which are not available from the initial publication of Climate-Fever. Therefore, we manually selected a subset of 102 claims (510 claim-evidence pairs) based on clarity of the claim formulation and balanced claim labels. Each sample was annotated on token-level by 3 annotators as either supporting the claim (label=1), contradicting the claim (label=-1) or neutral (label=0). Annotations were then averaged (rationale_numeric) and majority-voted (rationale_binary).
#column "label"
claim_label_dict = {
'🙌 SUPPORT': 1,
'👎 REFUTE/CONTRADICT': -1,
'🤷 NOT ENOUGH INFO': 0,
'🤨 DISPUTE (SUPPORT and REFUTE)': 2}
#column "evidence_labels"
evidence_labels_dict = {
0: "supports",
1: "refutes",
2: "not enough info"}
More details can be found in Stephanie Brandl\ & Oliver Eberle\ (2026) A Systematic Comparison between Extractive Self-Explanations and Human Rationales in Text Classification. Proceedings of the 6th Workshop on Trustworthy Natural Language Processing. (to appear)