ljvmiranda921/PolyglotTeachers-SFT-Synth-Data

<div style="display: flex; align-items: center; gap: 20px;"> <img alt="Logo for UCam" src="cambridgelogo.png" style="height: 80px; width: auto;"> <img alt="Logo for LTL" src="ltllogo2.svg" style="height: 80px; width: auto;"> </div>

Website: ljvmiranda921.github.io/polyglot-teachers/

PolyglotTeachers-SFT-Synth

This dataset contains synthetic supervised fine-tuning examples generated by the best teacher we found in the paper [Polyglot Teachers: Evaluating Language Models for Multilingual Synthetic Data Generation](), where we systematically characterize what makes a good teacher model. It contains examples across six languages: Arabic, Czech, German, Indonesian, Japanese, Spanish, and Tagalog. Note: In our experiments, we subsampled 10k examples per language for training. Here we release the full unfiltered set to enable reproducibility and give researchers the flexibility to construct their own subsamples or training mixtures.

Dataset Summary

• —Languages: Arabic (ar), Czech (cs), German (de), Indonesian (id), Japanese (ja), Spanish (es), Tagalog (tl)
• —Total examples: 315,596
• —Teacher model: google/gemma-3-27b-it
• —Generation strategies: generate, respond, translate

Language Distribution

Data Sources

The seed data comes from several multilingual datasets, which were then used to synthesize new examples via Gemma-3-27B-IT. Each source dataset was processed using one of three strategies: generate (create new prompt-response pairs from a seed), respond (generate a response given a prompt), or translate (translate an English example into a target language).

• —allenai/WildChat-4.8M: multilingual prompt-response pairs from real user interactions.
• —openai/gsm8k: math word problems (English, translated into target languages).
• —Magpie-Align/Magpie-Pro-300K-Filtered: general chat data (English, translated into target languages).
• —nvidia/Helpsteer3: multilingual preference data.
• —OpenAssistant/oasst2: multilingual assistant conversations.
• —utter-project/EuroBlocks-SFT-Synthetic-1124: European multilingual synthetic data.
• —CohereLabs/aya_collection: multilingual instruction data.
• —arbml/CIDAR: Arabic instruction data.
• —indonlp/cendol_collection_v2: Indonesian instruction data.

Dataset Structure

Each example contains the following fields:

Usage

from datasets import load_dataset

ds = load_dataset("ljvmiranda921/PolyglotTeachers-SFT-Synth", split="train")

# Filter by language
arabic_ds = ds.filter(lambda x: x["language"] == "ar")

# Use the messages field directly for SFT
print(arabic_ds[0]["messages"])

Acknowledgements

LJVM and AK acknowledge the support of the UKRI Frontier Grant EP/Y031350/1 (EQUATE). This work was performed using joint resources provided by the Cambridge Service for Data Driven Discovery (CSD3) EP/T022159/1 and the Isambard AI National AI Research Resource (AIRR) ST/AIRR/I-A-I/1023, and the Microsoft Research Grant. LJVM would also like to thank Songbo Hu, Chen Cecilia Liu, Millicent Ochieng, and Felermino Ali for helpful and productive discussions on the project.

Citation

@misc{miranda2026polyglotteachersevaluatinglanguage,
      title={Polyglot Teachers: Evaluating Language Models for Multilingual Synthetic Data Generation}, 
      author={Lester James V. Miranda and Ivan Vulić and Anna Korhonen},
      year={2026},
      eprint={2604.11290},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2604.11290}, 
}