opus-100
gemma2_opus100_de_en_mergedbert-jepa-infonce-bilingual-15lang-opus100-15lang-infonce-bilingual4b-opus100-manga-GGUFbert-jepa-infonce-monolingual-15lang-opus100-15lang-infonce-monolingualbert-jepa-infonce-monolingual-opus100-5lang-infonce-monolingualbert-jepa-infonce-bilingual-opus100-5lang-infonce-bilingualbert-jepa-infonce-opus100-5lang-infoncezh2en_opus_100_t5
Datasets
All datasets matching “opus-100”opus-100
Dataset Card for OPUS-100
Dataset Summary
OPUS-100 is an English-centric multilingual corpus covering 100 languages.
OPUS-100 is English-centric, meaning that all training pairs include English on either the source or target side. The corpus covers 100 languages (including English).
The languages were selected based on the volume of parallel data available in OPUS.
Supported Tasks and Leaderboards
Translation.
Languages
OPUS-100 contains… See the full description on the dataset page: https://huggingface.co/datasets/Helsinki-NLP/opus-100.opus100_envi
Opus100
Source: https://huggingface.co/datasets/opus100
Num examples:
1,000,000 (train)
2,000 (validation)
192,744 (test)
Language: English
from datasets import load_dataset
load_dataset("tdtunlp/opus100_envi")
Format for Translation task
def preprocess(
sample,
instruction_key="### Instruction:",
input_key="Input:", response_key="<|endofprompt|>",
end_key="<|endoftext|>",
en2vi=True,
):
if en2vi:
if random.random() < 0.5:… See the full description on the dataset page: https://huggingface.co/datasets/vietgpt/opus100_envi.opus-100
Dataset Card for OPUS-100
Dataset Summary
OPUS-100 is an English-centric multilingual corpus covering 100 languages.
OPUS-100 is English-centric, meaning that all training pairs include English on either the source or target side. The corpus covers 100 languages (including English).
The languages were selected based on the volume of parallel data available in OPUS.
Supported Tasks and Leaderboards
Translation.
Languages
OPUS-100… See the full description on the dataset page: https://huggingface.co/datasets/ds123DDD/opus-100.ur_opus100_processedbg_opus100_processedha_opus100_processed