sentence-transformers/parallel-sentences-global-voices

Dataset Card for Parallel Sentences - Global Voices

This dataset contains parallel sentences (i.e. English sentence + the same sentences in another language) for numerous other languages. Most of the sentences originate from the OPUS website. In particular, this dataset contains the Global Voices dataset.

Related Datasets

The following datasets are also a part of the Parallel Sentences collection:

• —parallel-sentences-europarl
• —parallel-sentences-global-voices
• —parallel-sentences-muse
• —parallel-sentences-jw300
• —parallel-sentences-news-commentary
• —parallel-sentences-opensubtitles
• —parallel-sentences-talks
• —parallel-sentences-tatoeba
• —parallel-sentences-wikimatrix
• —parallel-sentences-wikititles
• —parallel-sentences-ccmatrix

These datasets can be used to train multilingual sentence embedding models. For more information, see sbert.net - Multilingual Models.

Dataset Subsets

all subset

• —Columns: "english", "non_english"
• —Column types: str, str
• —Examples:

    {
      "english": "We're thrilled to be honored as the jury's choice for the Best Journalistic Blog in English.",
      "non_english": "تغمرنا السعادة بهذا التكريم باختيارنا أفضل مدونة صحفية بالإنجليزية."
    }

• —Collection strategy: Combining all other subsets from this dataset.
• —Deduplified: No

en-... subsets

• —Columns: "english", "non_english"
• —Column types: str, str
• —Examples:

    {
      "english": "Lisa Stone of Surfette was on the jury that chose our blog for the DW honor.",
      "non_english": "Lisa Stone, do Surfette, participou do júri que escolher o nosso blog para a honra."
    }

• —Collection strategy: Processing the raw data from parallel-sentences and formatting it in Parquet, followed by deduplication.
• —Deduplified: Yes