CoolFace
Datasetpublic

maikezu/iwslt2026-metrics-shared-train-dev

Metrics-Shared Task IWSLT 2026 Train and Dev Set This dataset contains the train and dev sets for the Speech Translation Metrics Shared Task at IWSLT 2026. More details about the shared task can be found on the IWSLT website. The dataset is primarily designed for research in speech translation quality estimation. Task Goal Given a speech sample and a system-generated translation, the goal is to estimate a score that reflects the translation quality.… See the full description on the dataset page: https://huggingface.co/datasets/maikezu/iwslt2026-metrics-shared-train-dev.

sourceHugging Facecc-by-nc-nd-4.0updated 4mo agoView on Hugging Face
3likes99downloads
Dataset Card

Metrics-Shared Task IWSLT 2026 Train and Dev Set

This dataset contains the train and dev sets for the Speech Translation Metrics Shared Task at IWSLT 2026. More details about the shared task can be found on the IWSLT website.

The dataset is primarily designed for research in speech translation quality estimation.

Task Goal

Given a speech sample and a system-generated translation, the goal is to estimate a score that reflects the translation quality.


Dataset Splits

train

Contains a mix of:

  • —IWSLT 2023 human annotations (details IWSLT 2023)
  • —Previous and following segments can be inferred from the doc_id feature.
  • —Human evaluators considered context of one previous and one following segment.
  • —WMT 2024 human annotations (details WMT 2024)
  • —Evaluated on segmented audio. The information on previous/following segments is not available.
  • —WMT 2025 human annotations (details WMT 2025)
  • —Evaluated on segmented audio. The information on previous/following segments is not available.

train_synthetic

Contains:

  • —SpeechQE data (details SpeechQE)
  • —Based on Common Voice.
  • —Automatically annotated (synthetic scores).

dev

Contains:

  • —IWSLT 2025 ACL Talks human annotations (details IWSLT 2025)
  • —Previous and following segments can be inferred from the doc_id feature.
  • —Human evaluators considered context of one previous and one following segment.

Features

ColumnTypeDescription
audioAudioThe speech waveform of the segment
audio_pathstringPath to the audio file
doc_idstringUnique identifier for the segment/document
src_textstringSource text
src_text_systemstringSource text system (e.g. human, ASR model)
src_langstringSource language code (e.g., en)
tgt_textstringTarget text translation
tgt_langstringTarget language code (e.g., de)
domainstringDomain or dataset source
tgt_systemstringTarget system or model used for translation
scorefloat64Human or synthetic evaluation score (0–1)

Citation

@inproceedings{adelani-etal-2026-iwslt,
    title     = {Speech Translation and Metrics in 2026: Findings of the IWSLT Campaign},
    author    = {
               Adelani, David Ifeoluwa
                and Anastasopoulos, Antonios
                and Agostinelli, Victor
                and Bentivogli, Luisa
                and Bojar, Ond{\v{r}}ej
                and Brati{\`e}res, Sebastien
                and Carpuat, Marine
                and Cattoni, Roldano
                and Cettolo, Mauro
                and Chen, Lizhong
                and Federico, Marcello
                and Gaido, Marco
                and Gupta, Mahendra
                and Han, HyoJung
                and Hatami, Ali
                and Javorsk{\'y}, David
                and Jeon, Yejin
                and Kasztelnik, Marek
                and Liu, Danni
                and Luu, Nam
                and Ma, Min
                and Mach{\'a}{\v{c}}ek, Dominik
                and Maltais, Marie
                and Matusov, Evgeny
                and Maurya, Chandresh Kumar
                and McCrae, John P.
                and Moslem, Yasmin
                and Nakamura, Satoshi
                and Negri, Matteo
                and Niehues, Jan
                and Ojha, Atul Kr.
                and Ouyang, Siqi
                and Papi, Sara
                and Pol{\'a}k, Peter
                and Retkowski, Fabian
                and Savoldi, Beatrice
                and Sikasote, Claytone
                and Sperber, Matthias
                and St{\"u}ker, Sebastian
                and Sudoh, Katsuhito
                and Turchi, Marco
                and Waibel, Alex
                and Wilken, Patrick
                and Zouhar, Vil{\'e}m
                and Z{\"u}fle, Maike
                },
    booktitle = {Proceedings of the 23rd International Conference on Spoken Language Translation (IWSLT 2026)},
    year      = {2026},
    address = "San Diego, California, US",
    publisher = "Association for Computational Linguistics",
}

Dataset Card Contact

Maike Züfle @maikezu