Datasetpublic

uzw/canada_ssa_gender_neutral_first_names

This is the official dataset for Beyond Binary Gender Labels: Revealing Gender Bias in LLMs through Gender-Neutral Name Predictions Name-based gender prediction has traditionally categorized individuals as either female or male based on their names, using a binary classification system. That binary approach can be problematic in the cases of gender-neutral names that do not align with any one gender, among other reasons. Relying solely on binary gender categories without recognizing… See the full description on the dataset page: https://huggingface.co/datasets/uzw/canada_ssa_gender_neutral_first_names.

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face

0likes34downloads

Dataset Card

This is the official dataset for Beyond Binary Gender Labels: Revealing Gender Bias in LLMs through Gender-Neutral Name Predictions

Name-based gender prediction has traditionally categorized individuals as either female or male based on their names, using a binary classification system. That binary approach can be problematic in the cases of gender-neutral names that do not align with any one gender, among other reasons. Relying solely on binary gender categories without recognizing gender-neutral names can reduce the inclusiveness of gender prediction tasks. We introduce an additional gender category, i.e., "neutral", to study and address potential gender biases in Large Language Models (LLMs).

In this Canada SSA dataset, gender-neutral names were rare before 2000 (less than five first names per year) but increased in recent years (after 2010), we sampled 273 names per gender for each year from 2013 to 2020.

For Dynamic Gender Labe Dataset, please visit this page.

Dataset Statistics

We split the dataset into train/val/test sets. We keep all three genders (male/female/neutral) balanced across all three sets. Please see below and the paper for more details of our curated datasets: image/png

Citation

Please cite the below paper if you intent to use our data for your research:

@inproceedings{you-etal-2024-beyond,
    title = "Beyond Binary Gender Labels: Revealing Gender Bias in {LLM}s through Gender-Neutral Name Predictions",
    author = "You, Zhiwen  and
      Lee, HaeJin  and
      Mishra, Shubhanshu  and
      Jeoung, Sullam  and
      Mishra, Apratim  and
      Kim, Jinseok  and
      Diesner, Jana",
    editor = "Fale{\'n}ska, Agnieszka  and
      Basta, Christine  and
      Costa-juss{\`a}, Marta  and
      Goldfarb-Tarrant, Seraphina  and
      Nozza, Debora",
    booktitle = "Proceedings of the 5th Workshop on Gender Bias in Natural Language Processing (GeBNLP)",
    month = aug,
    year = "2024",
    address = "Bangkok, Thailand",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.gebnlp-1.16",
    doi = "10.18653/v1/2024.gebnlp-1.16",
    pages = "255--268",
}

Contact Information

If you have any questions, please email zhiweny2@illinois.edu.