jvamvas/apertus-pretrain-romansh-backtranslated

Version of https://hf.co/datasets/swiss-ai/apertus-pretrain-romansh (monolingual split only) that includes MT-generated translations into German.

The intended purpose of this dataset is to train MT systems or LLMs on the task of idiom-specific German→Romansh translation. Note that the German translations in this dataset might contain errors, since they have been automatically generated by an MT system.

Composition of the dataset and Romansh data sources

See https://hf.co/datasets/swiss-ai/apertus-pretrain-romansh

Documentation of dataset fields

The following fields are copied from the source dataset at https://hf.co/datasets/swiss-ai/apertus-pretrain-romansh:

• —text, id, filename, language, language_script, source, file_path, dump, language_score, pii_count, url

Four fields have been added:

• —predicted_variety: Romansh variety predicted by our idiom classifier
• —german_backtranslation: Translation generated by MT system
• —romansh_score_dictionary: LID score (Romansh / non-Romansh) using a dictionary-based method. This dataset only retains samples with a score >= 0.5
• —romansh_score_langdetect: LID probability predicted by Langdetect based on a custom Romansh profile. This dataset retains samples with a score >= 0.95

MT system

We used a work-in-progress MT system developed by UZH / Lia Rumantscha for creating the German translations: https://hf.co/jvamvas/nllb-200-1.3B-rm-ct2

The system is based on NLLB (https://huggingface.co/facebook/nllb-200-distilled-1.3B) and has been fine-tuned on German--Romansh translation. A technical report on the MT system is found at https://arxiv.org/abs/2603.25489.

This Colab notebook provides example code for model usage: https://colab.research.google.com/drive/1mpOJO7T3dBaeZ5-Ij5-CIMvIXmLbaOvb?usp=sharing

Since the MT system has been trained in the sentence level, we perform sentence splitting and translate the sentences invidually.

Language classification

Romansh language classifiers (https://github.com/ZurichNLP/romansh-lid) were used to automatically classify the source idiom and to filter out non-Romansh text.

See this Colab notebook for usage examples: https://colab.research.google.com/drive/1mpOJO7T3dBaeZ5-Ij5-CIMvIXmLbaOvb?usp=sharing

Data filtering

Besides LID filtering, we also filtered out parallel samples with a character length ratio > 1.5. We also skipped 409 samples where our MT system ran out of GPU memory.