ruibrogandrade/Social_I_QA_PT-PT

Dataset Card for SocialIQA_PT-PT

Dataset Summary

This repository contains a European Portuguese (pt-PT) translation of Social IQa, a benchmark for social commonsense reasoning. Each example includes a short context about a social situation, a question, and three answer candidates. The task is to choose the most plausible answer about intents, reactions, or social outcomes. Load with:

import datasets
data = datasets.load_dataset("ruibrogandrade/Social_I_QA_PT-PT")

Supported Tasks and Leaderboards

• —Multiple-Choice Question Answering (social commonsense)
• —Choice selection / classification (accuracy)

Languages

• —European Portuguese (pt-PT) for translated fields
• —English originals are included for reference

Example

An example from the validation split:

{
  "context": "Cameron decided to have a barbecue and gathered her friends together.",
  "question": "How would Others feel as a result?",
  "answerA": "like attending",
  "answerB": "like staying home",
  "answerC": "a good friend to have",
  "label": 1,
  "context_translated": "Cameron decidiu fazer um churrasco e reuniu os seus amigos.",
  "question_translated": "Como é que os outros se sentiriam como resultado?",
  "answerA_translated": "vontade de ir",
  "answerB_translated": "vontade de ficar em casa",
  "answerC_translated": "uma boa amiga de se ter"
}

Dataset Structure

• —context (string): short social scenario (EN).
• —question (string): question about the scenario (EN).
• —answerA, answerB, answerC (string): three candidate answers (EN).
• —label (int): index of the correct answer in {1, 2, 3}.
• —context_translated, question_translated, answerA_translated, answerB_translated, answerC_translated (string): pt-PT translations.

Data Splits

• —Train: 1,120 examples
• —Validation: 788 examples
• —Test: not included

Dataset Creation

Curation & pipeline. Translated with a prompt-engineered LLM pipeline (TowerInstruct-13B, beam search). Fields were translated individually to preserve structure and avoid artefacts. Post-processing. Cleaning removed labels/prefixes (e.g., “Answer:”/“Resposta:”), normalized punctuation/whitespace, and truncated cases where an answer repeated the question. Heuristic filters removed degenerate translations (e.g., answers ending with question marks).

Additional Information

License: Same as the original Social IQa dataset; see the source dataset’s license terms.

Citation

@inproceedings{sap2019socialiqa,
  title     = {Social IQa: Commonsense Reasoning about Social Interactions},
  author    = {Sap, Maarten and Rashkin, Hannah and Chen, Derek and Le Bras, Ronan and Choi, Yejin},
  booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)},
  year      = {2019}
}

@mastersthesis{andrade2025,
  title  = {Benchmarking Portuguese via Machine Translation and LLMs},
  author = {Andrade, Rui Brogueira},
  school = {Faculdade de Engenharia da Universidade do Porto (FEUP)},
  year   = {2025},
  url    = {https://repositorio-aberto.up.pt/handle/10216/168804}
}