Datasetpublic

jvamvas/apertus-pretrain-romansh-backtranslated-sft

Subsampled version of https://huggingface.co/datasets/jvamvas/apertus-pretrain-romansh-backtranslated that has been processed as follows: Balanced subsampling to 30k samples (5k samples per variety) Samples with higher LID scores are prioritized Formatted as German-to-Romansh translation instruction pairs in prompt/completion format (prompt: Übersetze den folgenden Text nach {variety}:\n\n{german_backtranslation}; completion: the Romansh text) Normalized linebreaks to have clear text… See the full description on the dataset page: https://huggingface.co/datasets/jvamvas/apertus-pretrain-romansh-backtranslated-sft.

sourceHugging Facecc-by-4.0updated 6mo agoView on Hugging Face

0likes10downloads

Dataset Card

Subsampled version of https://huggingface.co/datasets/jvamvas/apertus-pretrain-romansh-backtranslated that has been processed as follows:

—Balanced subsampling to 30k samples (5k samples per variety)
—Samples with higher LID scores are prioritized
—Formatted as German-to-Romansh translation instruction pairs in prompt/completion format (prompt: Übersetze den folgenden Text nach {variety}:\n\n{german_backtranslation}; completion: the Romansh text)
—Normalized linebreaks to have clear text boundaries: re.sub(r"\n+", "\n", text)