coastalcph/fair-rationales

Dataset Card for "FairRationales"

Dataset Summary

We present a new collection of annotations for a subset of CoS-E [[1]](#1), DynaSent [[2]](#2), and SST [[3]](#3)/Zuco [[4]](#4) with demographics-augmented annotations, balanced across age and ethnicity. We asked participants to choose a label and then provide supporting evidence (rationales) based on the input sentence for their answer.

Existing rationale datasets are typically constructed by giving annotators 'gold standard' labels, and having them provide rationales for these labels. Instead, we let annotators provide rationales for labels they choose themselves. This lets them engage in the decision process, but it also acknowledges that annotators with different backgrounds may disagree on classification decisions. Explaining other people’s choices is error-prone [[5]](#5), and we do not want to bias the rationale annotations by providing labels that align better with the intuitions of some demographics than with those of others.

Our annotators are balanced across age and ethnicity for six demographic groups, defined by ethnicity {Black/African American, White/Caucasian, Latino/Hispanic} and age {Old, Young}. Therefore, we can refer to our groups as their cross-product: {BO, BY, WO, WY, LO, LY}.

Dataset Details

DynaSent

We re-annotate N=480 instances six times (for six demographic groups), comprising 240 instances labeled as positive, and 240 instances labeled as negative in the DynaSent Round 2 test set (see [[2]](#2)). This amounts to 2,880 annotations, in total. To annotate rationales, we formulate the task as marking 'supporting evidence' for the label, following how the task is defined by [[6]](#6). Specifically, we ask annotators to mark all the words, in the sentence, they think shows evidence for their chosen label.

>Our annotations:

negative 1555 | positive 1435 | no sentiment 470 Total 3460

Note that all the data is uploaded under a single 'train' split (read ## Uses for further details).

SST2

We re-annotate N=263 instances six times (for six demographic groups), which are all the positive and negative instances from the Zuco dataset of Hollenstein et al. (2018), comprising a mixture of train, validation and test set instances from SST-2, which should be removed from the original SST data before training any model*.

These 263 reannotated instances do not contain any instances originally marked as neutral (or not conveying sentiment) because rationale annotation for neutral instances is ill-defined. Yet, we still allow annotators to evaluate a sentence as neutral, since we do not want to force our annotators to provide rationales for positive and negative sentiment that they do not see.

*The Zuco data contains eye-tracking data for 400 instances from SST. By annotating some of these with rationales, we add an extra layer of information for future research.

>Our annotations:

positive 1027 | negative 900 | no sentiment 163 Total 2090

Note that all the data is uploaded under a single 'train' split (read ## Uses for further details).

CoS-E

We use the simplified version of CoS-E released by [[6]](#6).

We re-annotate N=500 instances from the CoS-E test set six times (for six demographic groups) and ask annotators to firstly select the answer to the question that they find most correct and sensible, and then mark words that justifies that answer. Following [[7]](#7), we specify the rationale task with a wording that should guide annotators to make short, precise rationale annotations:

‘For each word in the question, if you think that removing it will decrease your confidence toward your chosen label, please mark it.’

>Our annotations:

Total 3760

Note that all the data is uploaded under a single 'train' split (read ## Uses for further details).

Dataset Sources

<!-- Provide the basic links for the dataset. -->

• —Repository: https://github.com/terne/BeingRightforWhoseRight_Reasons
• —Paper: Being Right for Whose Right Reasons?

<a id="uses">## Uses</a>

<!-- Address questions around how the dataset is intended to be used. --> In our paper, we present a collection of three existing datasets (SST2, DynaSent and Cos-E) with demographics-augmented annotations to enable profiling of models, i.e., quantifying their alignment (or agreement) with rationales provided by different socio-demographic groups. Such profiling enables us to ask whose right reasons models are being right for and fosters future research on performance equality/robustness.

For each dataset, we provide the data under a unique 'train' split due to the current limitation of not being possible to upload a dataset with a single 'test' split. Note, however, that the original itended used of these collection of datasets was to test the quality & alignment of post-hoc explainability methods. If you use it following different splits, please clarify it to ease reproducibility of your work.

Dataset Structure

<!-- This section provides a description of the dataset fields, and additional information about the dataset structure such as criteria used to create the splits, relationships between data points, etc. -->

Dataset Creation

Curation Rationale

Terne Sasha Thorn Jakobsen, Laura Cabello, Anders Søgaard. Being Right for Whose Right Reasons? In the Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

Annotation process

We refer to our paper for further details on the data (Section 3), and specifically on the Annotation Process (Section 3.1) and Annotator Population (Section 3.2).

Who are the annotators?

Annotators were recruited via Prolific and consented to the use of their responses and demographic information for research purposes.

The annotation tasks were conducted through Qualtrics surveys. The exact surveys can be found here.

References

<a id="1">[1]</a> Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, and Richard Socher. 2019. Explain Yourself! Leveraging Language Models for Commonsense Reasoning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4932–4942, Florence, Italy. Association for Computational Linguistics.

<a id="2">[2]</a> Christopher Potts, Zhengxuan Wu, Atticus Geiger, and Douwe Kiela. 2021. DynaSent: A Dynamic Benchmark for Sentiment Analysis. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 2388–2404, Online. Association for Computational Linguistics.

<a id="3">[3]</a> Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631–1642, Seattle, Washington, USA. Association for Computational Linguistics.

<a id="4">[4]</a> Nora Hollenstein, Jonathan Rotsztejn, Marius Troendle, Andreas Pedroni, Ce Zhang, and Nicolas Langer. 2018. Zuco, a simultaneous eeg and eye-tracking resource for natural sentence reading. Scientific Data.

<a id="5">[5]</a> Kate Barasz and Tami Kim. 2022. Choice perception: Making sense (and nonsense) of others’ decisions. Current opinion in psychology, 43:176–181.

<a id="6">[6]</a> Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C. Wallace. 2019. Eraser: A benchmark to evaluate rationalized nlp models.

<a id="7">[7]</a> Cheng-Han Chiang and Hung-yi Lee. 2022. Reexamining human annotations for interpretable nlp.

Citation

<!-- If there is a paper or blog post introducing the dataset, the APA and Bibtex information for that should go in this section. -->

@inproceedings{thorn-jakobsen-etal-2023-right,
    title = "Being Right for Whose Right Reasons?",
    author = "Thorn Jakobsen, Terne Sasha  and
      Cabello, Laura  and
      S{\o}gaard, Anders",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.acl-long.59",
    doi = "10.18653/v1/2023.acl-long.59",
    pages = "1033--1054",
    abstract = "Explainability methods are used to benchmark the extent to which model predictions align with human rationales i.e., are {`}right for the right reasons{'}. Previous work has failed to acknowledge, however, that what counts as a rationale is sometimes subjective. This paper presents what we think is a first of its kind, a collection of human rationale annotations augmented with the annotators demographic information. We cover three datasets spanning sentiment analysis and common-sense reasoning, and six demographic groups (balanced across age and ethnicity). Such data enables us to ask both what demographics our predictions align with and whose reasoning patterns our models{'} rationales align with. We find systematic inter-group annotator disagreement and show how 16 Transformer-based models align better with rationales provided by certain demographic groups: We find that models are biased towards aligning best with older and/or white annotators. We zoom in on the effects of model size and model distillation, finding {--}contrary to our expectations{--} negative correlations between model size and rationale agreement as well as no evidence that either model size or model distillation improves fairness.",
}

Dataset Card Contact

Thanks to @lautel for adding this dataset.