vietgpt/opus100_envi

Opus100 Source: https://huggingface.co/datasets/opus100 Num examples: 1,000,000 (train) 2,000 (validation) 192,744 (test) Language: English from datasets import load_dataset load_dataset("tdtunlp/opus100_envi") Format for Translation task def preprocess( sample, instruction_key="### Instruction:", input_key="Input:", response_key="<|endofprompt|>", end_key="<|endoftext|>", en2vi=True, ): if en2vi: if random.random() < 0.5:… See the full description on the dataset page: https://huggingface.co/datasets/vietgpt/opus100_envi.