projecte-aina/wnli-ca

WNLI-ca

Table of Contents

• —Table of Contents
• —Dataset Description
• —Dataset Summary
• —Supported Tasks and Leaderboards
• —Languages
• —Dataset Structure
• —Data Instances
• —Data Fields
• —Data Splits
• —Dataset Creation
• —Curation Rationale
• —Source Data
• —Annotations
• —Personal and Sensitive Information
• —Considerations for Using the Data
• —Social Impact of Dataset
• —Discussion of Biases
• —Other Known Limitations
• —Additional Information
• —Dataset Curators
• —Licensing Information
• —Citation Information
• —Contributions

Dataset Description

• —Website: https://cs.nyu.edu/~davise/papers/WinogradSchemas/WS.html
• —Point of Contact: Language Technologies Unit

Dataset Summary

"A Winograd schema is a pair of sentences that differ in only one or two words and that contain an ambiguity that is resolved in opposite ways in the two sentences and requires the use of world knowledge and reasoning for its resolution. The schema takes its name from Terry Winograd." Source: The Winograd Schema Challenge.

The Winograd NLI dataset presents 855 sentence pairs, in which the first sentence contains an ambiguity and the second one a possible interpretation of it. The label indicates if the interpretation is correct (1) or not (0).

This dataset is a professional translation into Catalan of Winograd NLI dataset as published in GLUE Benchmark.

Both the original dataset and this translation are licenced under a Creative Commons Attribution 4.0 International License.

Supported Tasks and Leaderboards

Textual entailment, Text classification, Language Model.

Languages

The dataset is in Catalan (ca-ES)

Dataset Structure

Data Instances

Three tsv files.

Data Fields

• —index
• —sentence 1: first sentence of the pair
• —sentence 2: second sentence of the pair
• —label: relation between the two sentences:
• —0: the second sentence does not entail a correct interpretation of the first one (neutral)
• —1: the second sentence entails a correct interpretation of the first one (entailment)

Example

Data Splits

• —wnli-train-ca.csv: 636
• —wnli-dev-ca.csv: 72
• —wnli-test-shuffled-ca.csv: 146

Dataset Creation

Curation Rationale

We translated this dataset to contribute to the development of language models in Catalan, a low-resource language, and to allow inter-lingual comparisons.

Source Data

• —GLUE Benchmark site

Initial Data Collection and Normalization

This is a professional translation of WNLI dataset into Catalan, commissioned by BSC TeMU within the Projecte AINA.

For more information on how the Winograd NLI dataset was created, visit the webpage The Winograd Schema Challenge.

Who are the source language producers?

For more information on how the Winograd NLI dataset was created, visit the webpage The Winograd Schema Challenge.

Annotations

Annotation process

We comissioned a professional translation of WNLI dataset into Catalan.

Who are the annotators?

Translation was commisioned to a professional translator.

Personal and Sensitive Information

No personal or sensitive information included.

Considerations for Using the Data

Social Impact of Dataset

This dataset contributes to the development of language models in Catalan, a low-resource language.

Discussion of Biases

[N/A]

Other Known Limitations

[N/A]

Additional Information

Dataset Curators

Text Mining Unit (TeMU) at the Barcelona Supercomputing Center (bsc-temu@bsc.es).

This work was funded by the Departament de la Vicepresidència i de Polítiques Digitals i Territori de la Generalitat de Catalunya within the framework of Projecte AINA.

Licensing Information

This work is licensed under a <a rel="license" href="https://creativecommons.org/licenses/by/4.0/">CC Attribution 4.0 International License</a>.

Citation Information

@inproceedings{gonzalez-agirre-etal-2024-building-data,
    title = "Building a Data Infrastructure for a Mid-Resource Language: The Case of {C}atalan",
    author = "Gonzalez-Agirre, Aitor  and
      Marimon, Montserrat  and
      Rodriguez-Penagos, Carlos  and
      Aula-Blasco, Javier  and
      Baucells, Irene  and
      Armentano-Oller, Carme  and
      Palomar-Giner, Jorge  and
      Kulebi, Baybars  and
      Villegas, Marta",
    editor = "Calzolari, Nicoletta  and
      Kan, Min-Yen  and
      Hoste, Veronique  and
      Lenci, Alessandro  and
      Sakti, Sakriani  and
      Xue, Nianwen",
    booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://aclanthology.org/2024.lrec-main.231",
    pages = "2556--2566",
}

Contributions

[N/A]