CoolFace
Datasetpublic

RicardoRei/wmt-da-human-evaluation

Dataset Summary This dataset contains all DA human annotations from previous WMT News Translation shared tasks. The data is organised into 8 columns: lp: language pair src: input text mt: translation ref: reference translation score: z score raw: direct assessment annotators: number of annotators domain: domain of the input text (e.g. news) year: collection year You can also find the original data for each year in the results section… See the full description on the dataset page: https://huggingface.co/datasets/RicardoRei/wmt-da-human-evaluation.

sourceHugging Faceapache-2.0updated 4y agoView on Hugging Face
10likes205downloads
Dataset Card

Dataset Summary

This dataset contains all DA human annotations from previous WMT News Translation shared tasks.

The data is organised into 8 columns:

  • —lp: language pair
  • —src: input text
  • —mt: translation
  • —ref: reference translation
  • —score: z score
  • —raw: direct assessment
  • —annotators: number of annotators
  • —domain: domain of the input text (e.g. news)
  • —year: collection year

You can also find the original data for each year in the results section https://www.statmt.org/wmt{YEAR}/results.html e.g: for 2020 data: https://www.statmt.org/wmt20/results.html

Python usage:

python
from datasets import load_dataset
dataset = load_dataset("RicardoRei/wmt-da-human-evaluation", split="train")

There is no standard train/test split for this dataset but you can easily split it according to year, language pair or domain. E.g. :

python
# split by year
data = dataset.filter(lambda example: example["year"] == 2022)

# split by LP
data = dataset.filter(lambda example: example["lp"] == "en-de")

# split by domain
data = dataset.filter(lambda example: example["domain"] == "news")

Note that most data is from News domain.

Citation Information

If you use this data please cite the WMT findings from previous years: