projecte-aina/teca

1---2annotations_creators:3- expert-generated4language_creators:5- found6language:7- ca8license:9- cc-by-nc-nd-4.010multilinguality:11- monolingual12size_categories:13- unknown14source_datasets: []15task_categories:16- text-classification17task_ids:18- natural-language-inference19pretty_name: teca20dataset_info:21  features:22  - name: id23    dtype: string24  - name: premise25    dtype: string26  - name: hypothesis27    dtype: string28  - name: label29    dtype: int6430  splits:31  - name: train32    num_bytes: 323545833    num_examples: 1693034  - name: validation35    num_bytes: 40538036    num_examples: 211637  - name: test38    num_bytes: 40108139    num_examples: 211740  download_size: 273535841  dataset_size: 404191942configs:43- config_name: default44  data_files:45  - split: train46    path: data/teca_train.jsonl47  - split: validation48    path: data/teca_validation.jsonl49  - split: test50    path: data/teca_test.jsonl51---52 53# Dataset Card for TE-ca54 55 56## Dataset Description57 58- **Website:** https://zenodo.org/record/476145859- **Repository** [HuggingFace](https://huggingface.co/projecte-aina)60- **Paper:** [Are Multilingual Models the Best Choice for Moderately Under-resourced Languages? A Comprehensive Assessment for Catalan](https://arxiv.org/abs/2107.07903)61- **Point of Contact:** [Language Technologies Unit](langtech@bsc.es)62 63 64### Dataset Summary65 66TE-ca is a dataset of textual entailment in Catalan, which contains 21,163 pairs of premises and hypotheses, annotated according to the inference relation they have (implication, contradiction or neutral).67 68This is the second version of the dataset, released on 29/03/2025, where encoding mistakes from the initial release have been corrected in both the 'hypothesis' and 'premise' column. Check https://huggingface.co/datasets/projecte-aina/teca/blob/main/OLD/{SPLIT}_changes.csv for teh corresponding hcanges in each split.69 70This dataset was developed by [BSC TeMU](https://temu.bsc.es/) as part of [Projecte AINA](https://politiquesdigitals.gencat.cat/ca/economia/catalonia-ai/aina/), to enrich the [Catalan Language Understanding Benchmark (CLUB)](https://club.aina.bsc.es/). 71 72This work is licensed under an <a rel="license" href="https://creativecommons.org/licenses/by-nc-nd/4.0/">Attribution-NonCommercial-NoDerivatives 4.0 International License</a>.73 74### Supported Tasks and Leaderboards75 76Textual entailment, Text classification, Language Model77 78### Languages79 80The dataset is in Catalan (`ca-ES`).81 82## Dataset Structure83 84### Data Instances85 86Three JSON files, one for each split.87 88### Example:89 90<pre>91    92    {93        "id": 3247,94        "premise": "L'ONU adopta a Marràqueix un pacte no vinculant per les migracions",95        "hypothesis": "S'acorden unes recomanacions per les persones migrades a Marràqueix",96        "label": "0"97    },98    {99        "id": 2825,100        "premise": "L'ONU adopta a Marràqueix un pacte no vinculant per les migracions",101        "hypothesis": "Les persones migrades seran acollides a Marràqueix",102        "label": "1"103    },104    {105        "id": 2431,106        "premise": "L'ONU adopta a Marràqueix un pacte no vinculant per les migracions",107        "hypothesis": "L'acord impulsat per l'ONU lluny de tancar-se",108        "label": "2"109    },110</pre>111 112### Data Fields113 114- premise: text115- hypothesis: text related to the premise116- label: relation between premise and hypothesis:117    * 0: entailment118    * 1: neutral119    * 2: contradiction120    121### Data Splits122 123* dev.json: 2116 examples124* test.json: 2117 examples125* train.json: 16930 examples126 127## Dataset Creation128 129### Curation Rationale130 131We created this dataset to contribute to the development of language models in Catalan, a low-resource language.132 133### Source Data134 135Source sentences are extracted from the [Catalan Textual Corpus](https://doi.org/10.5281/zenodo.4519349) and from [VilaWeb](https://www.vilaweb.cat) newswire.136 137#### Initial Data Collection and Normalization138 13912000 sentences from the BSC [Catalan Textual Corpus](https://doi.org/10.5281/zenodo.4519349), together with 6200 headers from the Catalan news site [VilaWeb](https://www.vilaweb.cat), were chosen randomly. We filtered them by different criteria, such as length and stand-alone intelligibility. For each selected text, we commissioned 3 hypotheses (one for each entailment category) to be written by a team of native annotators. 140 141Some sentence pairs were excluded because of inconsistencies. 142 143#### Who are the source language producers?144 145The Catalan Textual Corpus corpus consists of several corpora gathered from web crawling and public corpora. More information can be found [here](https://doi.org/10.5281/zenodo.4519349).146 147[VilaWeb](https://www.vilaweb.cat) is a Catalan newswire.148 149### Annotations150 151#### Annotation process152 153We commissioned 3 hypotheses (one for each entailment category) to be written by a team of annotators. 154 155#### Who are the annotators?156 157Annotators are a team of native language collaborators from two independent companies.158 159### Personal and Sensitive Information160 161No personal or sensitive information included.162 163## Considerations for Using the Data164 165### Social Impact of Dataset166 167We hope this dataset contributes to the development of language models in Catalan, a low-resource language.168 169### Discussion of Biases170 171[N/A]172 173### Other Known Limitations174 175[N/A]176 177## Additional Information178 179### Dataset Curators180 181Text Mining Unit (TeMU) at the Barcelona Supercomputing Center (bsc-temu@bsc.es)182 183This work was funded by the [Departament de la Vicepresidència i de Polítiques Digitals i Territori de la Generalitat de Catalunya](https://politiquesdigitals.gencat.cat/ca/inici/index.html#googtrans(ca|en) within the framework of [Projecte AINA](https://politiquesdigitals.gencat.cat/ca/economia/catalonia-ai/aina).184 185 186### Licensing Information187 188This work is licensed under an <a rel="license" href="https://creativecommons.org/licenses/by-nc-nd/4.0/">Attribution-NonCommercial-NoDerivatives 4.0 International License</a>.189 190### Citation Information191 192```193 194@inproceedings{armengol-estape-etal-2021-multilingual,195    title = "Are Multilingual Models the Best Choice for Moderately Under-resourced Languages? {A} Comprehensive Assessment for {C}atalan",196    author = "Armengol-Estap{\'e}, Jordi  and197      Carrino, Casimiro Pio  and198      Rodriguez-Penagos, Carlos  and199      de Gibert Bonet, Ona  and200      Armentano-Oller, Carme  and201      Gonzalez-Agirre, Aitor  and202      Melero, Maite  and203      Villegas, Marta",204    booktitle = "Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021",205    month = aug,206    year = "2021",207    address = "Online",208    publisher = "Association for Computational Linguistics",209    url = "https://aclanthology.org/2021.findings-acl.437",210    doi = "10.18653/v1/2021.findings-acl.437",211    pages = "4933--4946",212}213 214```215 216[DOI](https://doi.org/10.5281/zenodo.4529183)217