CoolFace
Datasetpublic

agentlans/translation-quality

Multilingual Translation Quality Dataset This dataset provides multilingual text chunks translated into English, accompanied by automated quality evaluations generated by multiple large language models. Dataset Details Source Data: agentlans/HuggingFaceFW-finetranslations-100-languages-sample Target Language: English Content: Multilingual chunks mapped to their English translations alongside automated judge scores. Evaluation Methodology The… See the full description on the dataset page: https://huggingface.co/datasets/agentlans/translation-quality.

sourceHugging Facecc-by-4.0updated 18d agoView on Hugging Face
0likes98downloads
Dataset Card

Multilingual Translation Quality Dataset

This dataset provides multilingual text chunks translated into English, accompanied by automated quality evaluations generated by multiple large language models.

Dataset Details

Evaluation Methodology

The quality of each translated chunk was assessed using **Qwen/Qwen3.5-4B** and **google/gemma-4-E2B-it**. The final score for each entry represents the arithmetic mean of the two evaluator scores.

Evaluators utilized the following prompt to score translations on a 1 to 5 scale:

text
Rate this machine translation to English.

{"original": _____, "english": _____}

1 = Major meaning errors or omissions; unusable.
2 = Some meaning correct, but major errors or omissions.
3 = Main meaning correct; noticeable errors or unnatural wording.
4 = Meaning accurate and complete; only minor issues.
5 = Fully accurate, complete, fluent, and natural.

Compare English with the original. Prioritize meaning accuracy and completeness over style.

Output only one digit: 1, 2, 3, 4, or 5.

Example Row

json
{
  "original": "Jan Bergerlind\nPå våren i 1971, samtidig som Jan gikk på «Famous Artist’s School», ble hans første illustrasjon publisert i «Tidningen Sjömannen». Deretter fulgte flere kurs i kunstmaling etter levende modeller. Det ene oppdraget ga det andre, og i 1975 arbeidet Jan som illustratør på heltid. De fleste oppdragene kom fra aviser, bokforlag og reklamebyråer. Til oppdragsgivere i Skandinavia og øvrige Europa har han under årenes løp illustrert serier og noveller i tidsskrifter, malt bokomslag, reklamebrosjyrer, annonser, samlerserier, postkort og julekort etc. Originalen males med pensel i olje- og akrylmaling. Jan vil med sine motiver bevare den gamle tradisjonen og flere av hans motiver er i dag klassiske. Brødrene Flaarønning AS har samarbeidet med Bergerlind i en årreke, der Bergerlind har designet flere av motivene til våre populære julekuler som er malt på innsiden av glasset.",
  "translated": "Jan Bergerlind\nIn the spring of 1971, while Jan was attending \"Famous Artist's School,\" his first illustration was published in \"Tidningen Sjömannen.\" Subsequently, he took several courses in art painting from live models. One assignment led to another, and in 1975 Jan worked as an illustrator full-time. Most assignments came from newspapers, publishing houses, and advertising agencies. For clients in Scandinavia and the rest of Europe, he has illustrated comics and short stories in magazines over the years, painted book covers, advertising brochures, advertisements, collector's series, postcards, and Christmas cards, etc. The originals are painted with brush in oil and acrylic paints. Jan intends to preserve the old tradition with his motifs, and several of his motifs are now classic. Brødrene Flaarønning AS has collaborated with Bergerlind for many years, where Bergerlind has designed several of the motifs for our popular Christmas baubles which are painted on the inside of the glass.",
  "language": "nob_Latn",
  "qwen_score": 4,
  "gemma_score": 4,
  "score": 4
}

Limitations

[!WARNING] This dataset has not yet undergone rigourous human verification.
  • Evaluator Bias: The evaluation models may exhibit performance discrepancies across different language families (e.g., Qwen showing stronger baseline competence with Asian languages, particularly Chinese).
  • Metric Validation: Automated scores require benchmarking and validation against dedicated semantic and quality estimation models such as LaBSE and Unbabel/wmt22-cometkiwi-da.

Licence

Creative Commons Attribution 4.0 International (CC-BY-4.0)