ZurichNLP/20min-XD

20min-XD: A Comparable Corpus of Swiss News Articles

Dataset Summary

20min-XD (20 Minuten cross-lingual document-level) is a comparable corpus of Swiss news articles in German and French, collected from the online editions of 20 Minuten and 20 minutes between 2015 and 2024. The dataset consists of 15,000 semantically aligned German and French article pairs. Unlike parallel corpora, 20min-XD captures a broad spectrum of cross-lingual similarity, ranging from near-translations to related articles covering the same event. This dataset is intended for non-commercial research use only – please refer to the accompanying license/copyright notice for details.

Details

• —Languages: German (de), French (fr)
• —Available versions:
• —Document level: Contains full article pairs.
• —Size: 15,000 article pairs
• —Sentence level: Contains alignable sentences from the 15'000 article pairs. Note that all alignable sentences are contained in this dataset without considering a similarity threshold. We recommend, setting a similarity threshold (e.g.: 79) to filter for semantically similar sentences.
• —Size: 117,126 sentence pairs
• —The datasets contain the following information:
• —Document level:
• —``id_de/fr``: unique article ID
• —``score``: cosine similarity score
• —``content_id_de/fr``: content ID
• —``pubtime_de/fr``: time of publication
• —``article_link_de/fr``: link to the online news article
• —``medium_code_de/fr``: abbreviation for the name of the news outlet
• —``medium_name_de/fr``: full name of the news outlet
• —``language_de/fr``: language code
• —``char_count_de/fr``: character count of the content
• —``head_de/fr``: article head text
• —``subhead_de/fr``: article subhead text
• —``content_de/fr``: article content text

• —Sentence level:
• —``id_de/fr``: unique sentence ID
• —``score``: cosine similarity score
• —``sentence_id_de/fr``: document internal sentence ID
• —``aligned_article_id_de/fr``: document ID
• —``sentence_de/fr``: sentence
• —``char_count_de/fr``: character count of the sentence
• —``char_count_diff``: absolute difference between character counts of German and French sentence

Citation

If you use 20min-XD in your research, please cite:

@inproceedings{wastl-et-al-2025-20min,
    title = "20min-{XD}: A Comparable Corpus of {S}wiss News Articles",
    author = "Wastl, Michelle  and
      Vamvas, Jannis  and
      Calleri, Selena  and
      Sennrich, Rico",
    editor = {Gerber, Jonathan  and
      Cieliebak, Mark  and
      Tuggener, Don  and
      H{\"u}rlimann, Manuela},
    booktitle = "Proceedings of the 10th edition of the Swiss Text Analytics Conference",
    month = may,
    year = "2025",
    address = "Winterthur, Switzerland",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.swisstext-1.1/",
    pages = "1--10"
}