sentence-transformers/parallel-sentences-talks

Dataset Card for Parallel Sentences - Talks

This dataset contains parallel sentences (i.e. English sentence + the same sentences in another language) for numerous other languages. Most of the sentences originate from the OPUS website. In particular, this dataset contains the Talks dataset.

Related Datasets

The following datasets are also a part of the Parallel Sentences collection:

• —parallel-sentences-europarl
• —parallel-sentences-global-voices
• —parallel-sentences-muse
• —parallel-sentences-jw300
• —parallel-sentences-news-commentary
• —parallel-sentences-opensubtitles
• —parallel-sentences-talks
• —parallel-sentences-tatoeba
• —parallel-sentences-wikimatrix
• —parallel-sentences-wikititles
• —parallel-sentences-ccmatrix

These datasets can be used to train multilingual sentence embedding models. For more information, see sbert.net - Multilingual Models.

Dataset Subsets

all subset

• —Columns: "english", "non_english"
• —Column types: str, str
• —Examples:

    {
      'english': "See, the thing we're doing right now is we're forcing people to learn mathematics.",
      'non_english': 'حسناً ان ما نقوم به اليوم .. هو ان نجبر الطلاب لتعلم الرياضيات',
    }

• —Collection strategy: Combining all other subsets from this dataset.
• —Deduplified: No

en-... subsets

• —Columns: "english", "non_english"
• —Column types: str, str
• —Examples:

    {
      'english': "So I think practicality is one case where it's worth teaching people by hand.",
      'non_english': 'Ich denke, dass es sich aus diesem Grund lohnt, den Leuten das Rechnen von Hand beizubringen.',
    }

• —Collection strategy: Processing the raw data from parallel-sentences and formatting it in Parquet, followed by deduplication.
• —Deduplified: Yes