qanastek/EMEA-V3

EMEA-V3 : European parallel translation corpus from the European Medicines Agency

Table of Contents

• —[Dataset Card for [Needs More Information]](#dataset-card-for-needs-more-information)
• —Table of Contents
• —Dataset Description
• —Dataset Summary
• —Supported Tasks and Leaderboards
• —Languages
• —Dataset Structure
• —Data Instances
• —Data Fields
• —Data Splits
• —Dataset Creation
• —Curation Rationale
• —Source Data
• —Initial Data Collection and Normalization
• —Who are the source language producers?
• —Personal and Sensitive Information
• —Considerations for Using the Data
• —Other Known Limitations
• —Additional Information
• —Dataset Curators
• —Licensing Information
• —Citation Information

Dataset Description

• —Homepage: https://opus.nlpl.eu/EMEA.php
• —Repository: https://github.com/qanastek/EMEA-V3/
• —Paper: https://aclanthology.org/L12-1246/
• —Leaderboard: [Needs More Information]
• —Point of Contact: Yanis Labrak

Dataset Summary

EMEA-V3 is a parallel corpus for neural machine translation collected and aligned by Tiedemann, Jorg during the OPUS project.

Supported Tasks and Leaderboards

translation: The dataset can be used to train a model for translation.

Languages

In our case, the corpora consists of a pair of source and target sentences for all 22 different languages from the European Union (EU).

List of languages : Bulgarian (bg),Czech (cs),Danish (da),German (de),Greek (el),English (en),Spanish (es),Estonian (et),Finnish (fi),French (fr),Hungarian (hu),Italian (it),Lithuanian (lt),Latvian (lv),Maltese (mt),Dutch (nl),Polish (pl),Portuguese (pt),Romanian (ro),Slovak (sk),Slovenian (sl),Swedish (sv).

Load the dataset with HuggingFace

from datasets import load_dataset
dataset = load_dataset("qanastek/EMEA-V3", split='train', download_mode='force_redownload')
print(dataset)
print(dataset[0])

Dataset Structure

Data Instances

lang,source_text,target_text
bg-cs,EMEA/ H/ C/ 471,EMEA/ H/ C/ 471
bg-cs,ABILIFY,ABILIFY
bg-cs,Какво представлява Abilify?,Co je Abilify?
bg-cs,"Abilify е лекарство, съдържащо активното вещество арипипразол.","Abilify je léčivý přípravek, který obsahuje účinnou látku aripiprazol."
bg-cs,"Предлага се под формата на таблетки от 5 mg, 10 mg, 15 mg и 30 mg, като диспергиращи се таблетки (таблетки, които се разтварят в устата) от 10 mg, 15 mg и 30 mg, като перорален разтвор (1 mg/ ml) и като инжекционен разтвор (7, 5 mg/ ml).","Je dostupný ve formě tablet s obsahem 5 mg, 10 mg, 15 mg a 30 mg, ve formě tablet dispergovatelných v ústech (tablet, které se rozpustí v ústech) s obsahem 10 mg, 15 mg a 30 mg, jako perorální roztok (1 mg/ ml) nebo jako injekční roztok (7, 5 mg/ ml)."
bg-cs,За какво се използва Abilify?,Na co se přípravek Abilify používá?

Data Fields

lang : The pair of source and target language of type String.

source_text : The source text of type String.

target_text : The target text of type String.

Data Splits

Dataset Creation

Curation Rationale

For details, check the corresponding pages.

Source Data

<!-- #### Initial Data Collection and Normalization

ddd -->

Who are the source language producers?

Every data of this corpora as been uploaded by Tiedemann, Jorg on Opus.

Personal and Sensitive Information

The corpora is free of personal or sensitive information.

Considerations for Using the Data

Other Known Limitations

The nature of the task introduce a variability in the quality of the target translations.

Additional Information

Dataset Curators

_Hugging Face EMEA-V3_: Labrak Yanis, Dufour Richard (Not affiliated with the original corpus)

_OPUS : Parallel Data, Tools and Interfaces in OPUS_: Tiedemann, Jorg.

<!-- ### Licensing Information

ddd -->

Citation Information

Please cite the following paper when using this dataset.

@inproceedings{tiedemann-2012-parallel,
    title = Parallel Data, Tools and Interfaces in OPUS,
    author = {
      Tiedemann, Jorg
    },
    booktitle = "Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)",
    month = may,
    year = 2012,
    address = Istanbul, Turkey,
    publisher = European Language Resources Association (ELRA),
    url = http://www.lrec-conf.org/proceedings/lrec2012/pdf/463_Paper.pdf,
    pages = 2214--2218,
    abstract = This paper presents the current status of OPUS, a growing language resource of parallel corpora and related tools. The focus in OPUS is to provide freely available data sets in various formats together with basic annotation to be useful for applications in computational linguistics, translation studies and cross-linguistic corpus studies. In this paper, we report about new data sets and their features, additional annotation tools and models provided from the website and essential interfaces and on-line services included in the project.,
}