RJuro/eu_debates

1---2license: cc-by-nc-sa-4.03source_datasets:4- coastalcph/eu_debates5language_creators:6- found7multilinguality:8- multilingual9language:10- bg11- cs12- da13- de14- el15- en16- es17- et18- fi19- fr20- hr21- hu22- it23- lt24- lv25- mt26- nl27- pl28- pt29- ro30- sk31- sl32- sv33tags:34- politics35size_categories:36- 10K<n<100K37pretty_name: EU Debates (JSONL Conversion)38---39 40# Dataset Description41 42This dataset is a **conversion of the original [`coastalcph/eu_debates`](https://huggingface.co/datasets/coastalcph/eu_debates)** dataset released by [Chalkidis and Brandl (2024)](https://arxiv.org/abs/2403.13592).43 44The goal of this repository is to provide the same underlying data **without a Python loading script**, in a standard format (JSON Lines / Parquet) compatible with the current Hugging Face `datasets` library and automated data loading.45 46The original EU Debates corpus consists of approx. 87k individual speeches in the period 2009–2023.47The data was exhaustively scraped from the official European Parliament Plenary website ([link](https://www.europarl.europa.eu/)). All speeches are time-stamped, thematically organized in debates, and include metadata about:48 49- the speaker's identity (full name, euro-party affiliation, speaker role),50- the debate (date and title),51- language information, and (where available) machine-translated versions in English.52 53Older debate speeches are originally in English, while newer ones are linguistically diverse across the 23 official EU languages. Machine-translated English versions are provided using the EasyNMT framework with the [M2M-100 (418M)](https://huggingface.co/facebook/m2m100_418M) model (Fan et al., 2020).54 55This repository only changes the **storage format** (to `train.jsonl` / Parquet) and **removes the Python loading script**.56The data contents and fields are preserved from the original dataset.57 58# Data Fields59 60Each row / JSONL line is a single speech with the following fields:61 62- `speaker_name`: `string`, full name of the speaker.63- `speaker_party`: `string`, name of the euro-party (group) that the MEP is affiliated with.64- `speaker_role`: `string`, role of the speaker (e.g., Member of the European Parliament (MEP), EUROPARL President).65- `debate_title`: `string`, title of the debate in the European Parliament.66- `date`: `string`, full date of the speech in `YYYY-MM-DD` format.67- `year`: `string`, year of the speech in `YYYY` format.68- `intervention_language`: `string`, language code of the original intervention.69- `original_language`: `string`, language code of the original text.70- `text`: `string`, full original speech of the speaker.71- `translated_text`: `string` or `null`, machine translation of the speech into English if the original is not English, otherwise `null`.72 73# Data Instances74 75Example of a data instance:76 77```json78{79  "speaker_name": "Michèle Striffler",80  "speaker_party": "PPE",81  "speaker_role": "MEP",82  "debate_title": "Famine in East Africa (debate)",83  "date": "2011-09-15",84  "year": "2011",85  "intervention_language": "fr",86  "original_language": "fr",87  "text": "Monsieur le Président, Madame le Commissaire, chers collègues, la situation humanitaire sans précédent que connaît la Corne de l'Afrique continue [...]",88  "translated_text": "Mr. President, Mr. Commissioner, dear colleagues, the unprecedented humanitarian situation of the Horn of Africa continues [...]"89}90```91 92# How to Use93 94### From the Hugging Face Hub95 96If the dataset is hosted under `RJuro/eu_debates`:97 98```python99from datasets import load_dataset100 101eu_debates = load_dataset("RJuro/eu_debates", split="train")102```103 104### From Local Files105 106If you downloaded the `train.jsonl` file locally:107 108```python109from datasets import load_dataset110 111eu_debates = load_dataset(112    "json",113    data_files={"train": "train.jsonl"},114    split="train",115)116```117 118If you use Parquet instead:119 120```python121from datasets import load_dataset122 123eu_debates = load_dataset(124    "parquet",125    data_files={"train": "train.parquet"},126    split="train",127)128```129 130# Dataset Statistics131 132The statistics below are inherited from the original `coastalcph/eu_debates` dataset.133 134### Distribution of speeches across euro-parties:135 136| Euro-party  | No. of Speeches |137|-------------|-----------------|138| EPP         | 25,455 (29%)    |139| S&D         | 20,042 (23%)    |140| ALDE        | 8,946 (10%)     |141| ECR         | 7,493 (9%)      |142| ID          | 6,970 (8%)      |143| GUE/NGL     | 6,780 (8%)      |144| Greens/EFA  | 6,398 (7%)      |145| NI          | 5,127 (6%)      |146| **Total**   | **87,221**      |147 148### Distribution of speeches across years and euro-parties:149 150| Year | EPP | S&D | ALDE | ECR | ID | GUE/NGL | Greens/EFA | NI | Total |151|---|---|---|---|---|---|---|---|---|---|152| 2009 | 748 | 456 | 180 | 138 | 72 | 174 | 113 | 163 | **2044** |153| 2010 | 3205 | 1623 | 616 | 340 | 341 | 529 | 427 | 546 | **7627** |154| 2011 | 4479 | 2509 | 817 | 418 | 761 | 792 | 490 | 614 | **10880** |155| 2012 | 3366 | 1892 | 583 | 419 | 560 | 486 | 351 | 347 | **8004** |156| 2013 | 724 | 636 | 240 | 175 | 152 | 155 | 170 | 154 | **2406** |157| 2014 | 578 | 555 | 184 | 180 | 131 | 160 | 144 | 180 | **2112** |158| 2015 | 978 | 1029 | 337 | 405 | 398 | 325 | 246 | 240 | **3958** |159| 2016 | 919 | 972 | 309 | 387 | 457 | 317 | 225 | 151 | **3737** |160| 2017 | 649 | 766 | 181 | 288 | 321 | 229 | 162 | 135 | **2731** |161| 2018 | 554 | 611 | 161 | 242 | 248 | 175 | 160 | 133 | **2284** |162| 2019 | 1296 | 1339 | 719 | 556 | 513 | 463 | 490 | 353 | **5729** |163| 2020 | 1660 | 1564 | 823 | 828 | 661 | 526 | 604 | 346 | **7012** |164| 2021 | 2147 | 2189 | 1290 | 1062 | 909 | 708 | 990 | 625 | **9920** |165| 2022 | 2436 | 2273 | 1466 | 1177 | 827 | 962 | 1031 | 641 | **10813** |166| 2023 | 1716 | 1628 | 1040 | 878 | 619 | 779 | 795 | 499 | **7954** |167 168### Distribution of speeches across the 23 EU official languages:169 170| Language | No. of Speeches |171|----------|-----------------|172| en       | 40,736 (46.7%)  |173| de       | 6,497 (7.5%)    |174| fr       | 6,024 (6.9%)    |175| es       | 5,172 (5.9%)    |176| it       | 4,506 (5.2%)    |177| pl       | 3,792 (4.4%)    |178| pt       | 2,713 (3.1%)    |179| ro       | 2,308 (2.7%)    |180| el       | 2,290 (2.6%)    |181| nl       | 2,286 (2.6%)    |182| hu       | 1,661 (1.9%)    |183| hr       | 1,509 (1.7%)    |184| cs       | 1,428 (1.6%)    |185| sv       | 1,210 (1.4%)    |186| bg       | 928 (1.1%)      |187| sk       | 916 (1.1%)      |188| sl       | 753 (0.9%)      |189| fi       | 693 (0.8%)      |190| lt       | 618 (0.7%)      |191| da       | 578 (0.7%)      |192| et       | 342 (0.4%)      |193| lv       | 184 (0.2%)      |194| mt       | 0 (0.0%)        |195 196# Citation Information197 198If you use this dataset, please cite the original work:199 200> Llama meets EU: Investigating the European political spectrum through the lens of LLMs.201> Ilias Chalkidis and Stephanie Brandl.202> In the Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL),203> Mexico City, Mexico, June 16–21, 2024.204 205```bibtex206@inproceedings{chalkidis-and-brandl-eu-llama-2024,207    title = "Llama meets EU: Investigating the European political spectrum through the lens of LLMs",208    author = "Chalkidis, Ilias  and Brandl, Stephanie",209    booktitle = "Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics",210    month = jun,211    year = "2024",212    address = "Mexico City, Mexico",213    publisher = "Association for Computational Linguistics",214}215```216 217This repository only provides a format-converted, script-free version of the original dataset; all credit for data collection and annotation goes to the original authors.