datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
fineweb-edu-translated
Helsinki-NLP/fineweb-edu-translated
fineweb-edu-tanslated is a collection of automatically translated documents from fineweb-edu.
Translations are based on OPUS-MT and HPLT-MT models.
The data in v1.0 covers 36,704,000 documents with over 28 billion space-searated tokens of English data translated into 36 languages.
The total v1.0 data set includes over 960 billion tokens and the translated documents are aligned across all languages.
In the v1.1 release, additional translations… See the full description on the dataset page: https://huggingface.co/datasets/Helsinki-NLP/fineweb-edu-translated.nemotron-cc-translated
Helsinki-NLP/nemotron-cc-translated
nemotron-cc-tanslated is a collection of automatically translated documents from nemotron-cc taken out of the high-quality subset.
Translations are based on OPUS-MT and HPLT-MT models.
The data in v1.0 covers 156,431,999 documents with over 70 billion space-searated tokens of English data translated into 36 languages.
The total v1.0 data set includes over 2.4 trillion tokens and the translated documents are aligned across all languages.
v1.1… See the full description on the dataset page: https://huggingface.co/datasets/Helsinki-NLP/nemotron-cc-translated.Dolci-Instruct-SFT-translatedTranslatePsy-AfriSLM-Synthetic-Mix
TranslatePsy-AfriSLM Synthetic Mix
TranslatePsy-AfriSLM Synthetic Mix is a quality-filtered synthetic parallel corpus for machine translation between English and 19 Sub-Saharan African languages. It contains 215,653,192 bidirectional training examples and was selected as the primary African translation component used to post-train the TranslatePsy-AfriSLM model family.
The dataset accompanies the EMNLP 2026 paper TranslatePsy-AfriSLM: High-Quality Data Scaling For Low-Resource… See the full description on the dataset page: https://huggingface.co/datasets/qvac/TranslatePsy-AfriSLM-Synthetic-Mix.Dolci-Think-SFT-translated
Dolci-Think-SFT-translated
Machine translations of the Dolci-Think-SFT-32B dataset, produced with gemma-4-31B-it. The samples selected for translation are those where content_quality == "excellent" according to the propella annotations.
Columns
Each row is a translated conversation plus the result of a post-translation quality filter:
id — source record id.
messages — the translated conversation (list of {content, role}).
filter_pass — true if the row passed… See the full description on the dataset page: https://huggingface.co/datasets/openeurollm/Dolci-Think-SFT-translated.Dolci-Instruct-DPO-translatedsmoltalk-ko-translate
번역 결과에 토큰이 반복된 결과들이 포함되어 있습니다. 필터링 후 재업로드 하겠습니다.
Z 알고리즘을 사용해 결과를 필터링 하였으며 {subset}_filtered 로 업로드하였습니다.
필터링 후 결과
subset
전
후
split/train
4205413
4162254
split/test
221249
218830
merge/train
1043917
1034473
merge/test
54948
54430
HuggingFaceTB/smoltalk 데이터셋의 subset:all을 nayohan/llama3-instrucTrans-enko-8b 모델을 사용해 번역했습니다.
원본의 messages 중 4096 token 이 넘어가는 content가 있다면 해당 레코드는 번역하지 않았습니다.
DeepSeek-R1-Distilled-Translate-en-zh_CN-39k
DeepSeek R1 满血蒸馏英中翻译数据集
本数据集是一个专门用于微调语言模型的英中翻译数据集，主要通过DeepSeek R1满血版蒸馏完成。
SFT训练版本
为了方便直接进行监督微调（Supervised Fine-tuning，SFT）训练，我们提供了两个使用标准 instruction-input-output 格式预的处理版本：
带 CoT 版本
保留了翻译过程中的思维链（Chain of Thought）
适合训练具有推理能力的翻译模型
无 CoT 版本
移除了思维链部分，只保留最终翻译结果
更适合训练直接输出翻译结果的模型
数据更简洁，训练更高效
项目依赖
本项目主要基于以下工具完成数据处理和生成：
llm-tools: 用于大语言模型数据处理的工具集合
qa-generator: 基于大语言模型的问答数据生成工具
数据集概览
关键统计
总样本数：38,981
数据集结构
字段说明… See the full description on the dataset page: https://huggingface.co/datasets/FradSer/DeepSeek-R1-Distilled-Translate-en-zh_CN-39k.DeepSeek-R1-Distilled-Translate-en-zh_CN-39k-Alpaca-GPT4
DeepSeek R1 满血蒸馏英中翻译数据集 Alpaca GPT-4（带 CoT 版本）
本数据集是 @FradSer/DeepSeek-R1-Distilled-Translate-en-zh_CN-39k 的 Alpaca GPT-4 版本，专门用于微调语言模型的英中翻译任务。采用标准的指令微调格式，更适合直接用于 SFT（Supervised Fine-tuning）训练。
本项目主要基于以下工具完成数据处理和生成：
llm-tools: 用于大语言模型数据处理的工具集合
数据集概览
关键统计
总样本数：38,981
数据集结构
字段说明
features:
- name: instruction # 待翻译的英文文本
dtype: string
- name: input # 空字符串，保持与标准指令格式一致
dtype: string
- name: output #… See the full description on the dataset page: https://huggingface.co/datasets/FradSer/DeepSeek-R1-Distilled-Translate-en-zh_CN-39k-Alpaca-GPT4.ai_society_translated
CAMEL: Communicative Agents for “Mind” Exploration of Large Scale Language Model Society
Github: https://github.com/lightaime/camel
Website: https://www.camel-ai.org/
Arxiv Paper: https://arxiv.org/abs/2303.17760
Dataset Summary
The original AI Society dataset is in English and is composed of 25K conversations between two gpt-3.5-turbo agents. The dataset is obtained by running role-playing for a combination of 50 user roles and 50 assistant roles with each… See the full description on the dataset page: https://huggingface.co/datasets/camel-ai/ai_society_translated.nemotron-cc-10K-sample-translated
Translated Nemotron-cc-hq samples
This dataset contains translated samples from https://huggingface.co/datasets/spyysalo/nemotron-cc-10K-sample
Currently, the following are available, we will add other models and languages:
Model
Languages
Gemma-3-4b-it
["Bulgarian", "Czech", "Danish", "German", "Estonian", "Finnish", "French", "Croatian", "Dutch"]
EuroLLM-9B-Instruct
["Bulgarian", "Czech", "Danish", "German", "Greek", "Estonia", "Finnish", "French", "Irish"… See the full description on the dataset page: https://huggingface.co/datasets/openeurollm/nemotron-cc-10K-sample-translated.bhasha-wiki-translated
Bhasha Wikipedia Translated
Translated wikipedia articles
Dataset Details
Dataset is being updated
Dataset Description
We have translated 6.185 million English wikipedia articles into 6 Indic languages. The translations were done using IndicTrans2 model.
Curated by: Soket AI labs
Language(s) (NLP): Hindi, Bengali, Gujarati, Tamil, Kannada, Urdu
License: cc-by-sa-4.0
Uses
For pretraining or Fine tuning for Indic language models
Dataset… See the full description on the dataset page: https://huggingface.co/datasets/soketlabs/bhasha-wiki-translated.pixiv-dic-auto-translated
KaraKaraWitch/pixiv-dic-auto-translated
Unfiltered Japanese→English machine translation of the Pixiv Encyclopedia.
Dataset Summary
~150k Pixiv Encyclopedia head-words and article snippets, raw-translated into English.
Translations are public-domain. No blacklist, no bowdlerisation—if the JP page existed, it’s in here.
Languages
Code
Language
Role
ja
Japanese
original
en
English
machine target
The language: list only contains ja and en because… See the full description on the dataset page: https://huggingface.co/datasets/KaraKaraWitch/pixiv-dic-auto-translated.Vietnamese-Salesforce-xlam-function-calling-60k-gg-translatedVietnamese-nampdn-ai-tiny-webtext-gg-translatedimmersive_translate_en-zh
MiniCPM5-1B Immersive Translation SFT Dataset
英译中微调数据集，专为沉浸式翻译插件场景设计。用于微调 MiniCPM5-1B-Base，使其在插件运行时稳定遵循翻译规则、保留代码与 HTML 格式、正确处理多段 %% 分隔。
数据集描述
本数据集主要训练以下能力：
严格遵循沉浸式翻译 system prompt 中的 5 条翻译规则
多段输入的 %% 段落分隔，输入输出段落数严格一致
代码块、行内代码、HTML 标签、URL、专有名词的原样保留
技术文档（GitHub README、Hugging Face 文档）与学术摘要（arXiv）的英译中
单段输入直接输出译文，无"翻译："等额外前缀
数据格式为 ShareGPT 对话格式，每条样本包含 system / user / assistant 三角色。
数据来源
来源
说明
原始规模
本数据集采样量
License
Mxode/BiST… See the full description on the dataset page: https://huggingface.co/datasets/Variable65536/immersive_translate_en-zh.translate_tinystories_dialogue_envi
Description
The dataset is from vilm/tinystories-envi, formatted as dialogues for speed and ease of use. Many thanks to vilm for releasing it.
Importantly, this format is easy to use via the default chat template of transformers, meaning you can use huggingface/alignment-handbook immediately, unsloth.
Structure
Data is created from "English - Vietnamese" or "Vietnamese - English" translation data pairs with prompts to specify for the model.
Here is a sample:
[
{… See the full description on the dataset page: https://huggingface.co/datasets/lamhieu/translate_tinystories_dialogue_envi.yeji-bazi-translated-ko
██████╗ █████╗ ███████╗██╗ ████████╗██████╗ █████╗ ███╗ ██╗███████╗
██╔══██╗██╔══██╗╚══███╔╝██║ ╚══██╔══╝██╔══██╗██╔══██╗████╗ ██║██╔════╝
██████╔╝███████║ ███╔╝ ██║ ██║ ██████╔╝███████║██╔██╗ ██║███████╗
██╔══██╗██╔══██║ ███╔╝ ██║ ██║ ██╔══██╗██╔══██║██║╚██╗██║╚════██║
██████╔╝██║ ██║███████╗██║ ██║ ██║ ██║██║ ██║██║ ╚████║███████║
╚═════╝ ╚═╝ ╚═╝╚══════╝╚═╝ ╚═╝ ╚═╝ ╚═╝╚═╝ ╚═╝╚═╝ ╚═══╝╚══════╝
⚡ MASSIVE TRANSLATION CORPUS… See the full description on the dataset page: https://huggingface.co/datasets/tellang/yeji-bazi-translated-ko.sow-translate
sow-translate
A large translation corpus, shipped as plain JSONL plus a pre-tokenised binary pair.
Commercial use requires a revenue-sharing agreement — see Licence below.
file
content
jsonl/sow-translate-NN-of-10.jsonl
the corpus as text, one {"text": ...} object per line
sow-translate.bin
the same corpus pre-tokenised, uint16 token ids (30,260,331,570 B)
sow-translate.idx
MMIDIDX v1 offset index for the above (2,272,119,922 B)
Statistics… See the full description on the dataset page: https://huggingface.co/datasets/tritueviet/sow-translate.marathi-alpaca-cleaned-translated
Marathi Alpaca Cleaned Translated
A Marathi translation of the 51,760-row Alpaca-Cleaned instruction-tuning dataset — Unsloth's hosted fork of yahma/alpaca-cleaned, which fixes hallucinations, empty outputs, and formatting errors found in the original Stanford Alpaca-52k dataset.
Translated using Meta's facebook/nllb-200-distilled-600M model. Built to reproduce and evaluate the Marathi instruction-tuning experiment from Khade et al., CHiPSAL 2025. The original paper translated… See the full description on the dataset page: https://huggingface.co/datasets/lubzo/marathi-alpaca-cleaned-translated.Dolci-Instruct-SFT-translated
Dolci-Instruct-SFT-translated (Swedish)
This dataset is a Swedish machine translation of the openeurollm/Dolci-Instruct-SFT-translated dataset, originally created as part of the OpenEuroLLM project.
Dataset details
Examples: 494,841 multi-turn conversations
Language: Swedish (sv-SE)
Format: Chat/messages format (id, messages)
License: Apache 2.0
Translation
All English source texts were machine-translated to Swedish using Google Gemma 3 27B-IT (w8a8_fp8… See the full description on the dataset page: https://huggingface.co/datasets/AI-Sweden-Models/Dolci-Instruct-SFT-translated.FairytaleQA-translated-spanish
Dataset Card for FairytaleQA-translated-ptBR
Dataset Summary
This repository contains the Spanish machine-translated version of the original English FairytaleQA dataset (https://huggingface.co/datasets/WorkInTheDark/FairytaleQA). FairytaleQA is an open-source dataset designed to enhance comprehension of narratives, aimed at students from kindergarten to eighth grade. The dataset is meticulously annotated by education experts following an evidence-based theoretical… See the full description on the dataset page: https://huggingface.co/datasets/benjleite/FairytaleQA-translated-spanish.gsm8k-translated
Multilingual GSM8K Translations
This dataset contains machine-translated versions of GSM8K in these languages:
French (fr)
German (de)
Hindi (hi)
Dataset Structure
For each language, we provide the original GSM8K train and test splits:
train: 7,473 samples
test: 1,319 samples
Each sample consists of a question and an answer.
The question describes a grade-school-level math word problem that requires multi-step mathematical reasoning. The answer contains a… See the full description on the dataset page: https://huggingface.co/datasets/math-across-languages/gsm8k-translated.bluemoon-fandom-1-1-rp-jp-translated
bluemoon-fandom-1-1-rp-jp-translated
A subset of Squish42/bluemoon-fandom-1-1-rp-cleaned translated to Japanese using command-r-08-2024.
Misc. info
I used openrouter's api for inference with command-r-08-2024. Doing so is roughly 4x quicker than running the model locally, doesn't use up 95% of my vram, and doesn't make my 3090 as loud as my neighbours.
I decided to use command-r-08-2024 because it is completely uncensored for nsfw translation and provides translation… See the full description on the dataset page: https://huggingface.co/datasets/joujiboi/bluemoon-fandom-1-1-rp-jp-translated.stanford-alpaca-cleaned-turkish-translated09/04/2023 Update:
New instructions added from: https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM
Original Version: https://github.com/tatsu-lab/stanford_alpaca#data-release
AI BASED TRANSLATION RESULTS OF STANFORD ALPACA EN TO TR
For academic only, please cite before you use it.
Taşar, D. E. T. (2023). stanford-alpaca-cleaned-turkish-translated [Dataset]. In Stanford Alpaca TR (1.0.1.a). https://huggingface.co/datasets/emre/stanford-alpaca-cleaned-turkish-translated… See the full description on the dataset page: https://huggingface.co/datasets/emre/stanford-alpaca-cleaned-turkish-translated.translated-babylm-telugu
Translated BabyLM — Telugu (translated-babylm-telugu)
Dataset Description
This dataset is a Telugu translation of the English BabyLM 2026 corpus, produced using IndicTrans2, a state-of-the-art neural machine translation model developed by AI4Bharat for Indic languages. The dataset is intended for training and evaluating language models on Telugu, following the BabyLM challenge setup.
Translated by: IndicTrans2 (ai4bharat/indictrans2-en-indic-1B)
Source language:… See the full description on the dataset page: https://huggingface.co/datasets/pulipakav-1/translated-babylm-telugu.Open_o1_sft_Pro_translated_jp
概要
このデータセットはOpen_o1_sft_ProデータセットをQwen社のQwen2.5-14B-Instructを用いて日本語に翻訳したものになります。
テンプレート
テンプレートは以下です。
{"conversations": [{"role": "user", "content": "入力"}, {"role": "assistant", "thought": "思考",
"content": "出力"}, ...],
"id": id(整数),
"dataset": "元データセットの名前"}
ライセンス
ライセンスは元データセットに準じます。
謝辞
データセットの製作者様,Qwenの開発者様,計算資源を貸してくださったVolt mindの皆様に感謝します。
task1617_cc_alligned_translate_tel_eng
Dataset Card for Natural Instructions (https://github.com/allenai/natural-instructions) Task: task1617_cc_alligned_translate_tel_eng
Additional Information
Citation Information
The following paper introduces the corpus in detail. If you use the corpus in published work, please cite it:
@misc{wang2022supernaturalinstructionsgeneralizationdeclarativeinstructions,
title={Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP… See the full description on the dataset page: https://huggingface.co/datasets/Lots-of-LoRAs/task1617_cc_alligned_translate_tel_eng.hermes-3-dataset-ru-translated-prompts
Переведенные промты из hermes-3-dataset
Модель-переводчик Gemma-3-27b-it.
Переведены все промты.
Multi-turn промты переведены с учетом контекста англоязычного ответа.
Будет полезно для создания крупных русскоязычных инструктивных датасетов или Online RL.
Translated prompts from hermes-3-dataset
Translator model: Gemma-3-27b-it.
All prompts have been translated.
Multi-turn prompts were translated considering the context of the English response.
This will be useful… See the full description on the dataset page: https://huggingface.co/datasets/kristaller486/hermes-3-dataset-ru-translated-prompts.translated-babylm-hindi
Translated BabyLM — Hindi (translated-babylm-hindi)
Dataset Description
This dataset is a Hindi translation of the English BabyLM 2026 corpus, produced using IndicTrans2, a state-of-the-art neural machine translation model developed by AI4Bharat for Indic languages. The dataset is intended for training and evaluating language models on Hindi, following the BabyLM challenge setup.
Translated by: IndicTrans2 (ai4bharat/indictrans2-en-indic-1B)
Source language:… See the full description on the dataset page: https://huggingface.co/datasets/pulipakav-1/translated-babylm-hindi.