datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
gsm8k-multilingual-reasoning
gsm8k-multilingual-reasoning
GSM8K with reasoning translated to multiple languages
Schema
{"prompt": "...", "answer": "...", "reasoning": "...", "metadata": {...}}
Usage
from datasets importload_dataset
ds = load_dataset("eddie-OB/gsm8k-multilingual-reasoning")
print(ds["train"][0])
Source
Derived from OpenAI GSM8K.
gsm8k-multilingual
gsm8k-multilingual
GSM8K translated to multiple languages (no reasoning)
Schema
{"prompt": "...", "answer": "...", "metadata": {...}}
Usage
from datasets import load_dataset
ds = load_dataset("eddie-OB/gsm8k-multilingual")
print(ds["train"][0])
Source
Derived from OpenAI GSM8K.
multilingual-queries-2026
MAST Multilingual Queries 2026
This dataset contains the multilingual query set for MAST @ FIRE 2026, the Multilingual Agentic Search Track. MAST evaluates whether multilingual agentic search systems can answer complex questions posed in different languages.
MAST builds on BrowseComp-Plus (ACL 2026), a reproducible and verifiable extension of BrowseComp with challenging English queries, a verified English corpus of roughly 100K web-sourced documents, and human judgments. In the… See the full description on the dataset page: https://huggingface.co/datasets/mast-benchmark/multilingual-queries-2026.RAG_Multilingual
Dataset Card for RAG_Multilingual
Dataset Summary
RAG_Multilingual is an instruction-following synthetic QA dataset created from extractive QA datasets from Catalan, English and Spanish reference sets.
The reference datasets were: SQAD (https://huggingface.co/datasets/rajpurkar/squad), Catalanqa (https://huggingface.co/datasets/projecte-aina/catalanqa) and SQAC (https://huggingface.co/datasets/PlanTL-GOB-ES/SQAC).
This dataset, of 56.406 instances, was created by… See the full description on the dataset page: https://huggingface.co/datasets/projecte-aina/RAG_Multilingual.multilingual_qaMultilingual instructional dataset for extractive QA finetunning for Catalan, Spanish and English, using SQAD, SQAC, CatalanQA and COQCAT datasets.
Contains almost 55K questions, answers and contexts, with development and train splits.
cs_squad-3.0
Dataset Card for Czech Simple Question Answering Dataset 3.0
This a processed and filtered adaptation of an existing dataset. For raw and larger dataset, see Dataset Source section.
Dataset Description
The data contains questions and answers based on Czech wikipeadia articles.
Each question has an answer (or more) and a selected part of the context as the evidence.
A majority of the answers are extractive - i.e. they are present in the context in the exact form. The… See the full description on the dataset page: https://huggingface.co/datasets/fewshot-goes-multilingual/cs_squad-3.0.AutoMemoryBench
AutoMemoryBench
State-Contract Evaluation for Auditable Agent Memory
AutoMemoryBench evaluates whether an agent uses the right memory, and only
the admissible memory, under a query-time state contract. Each executable
contract partitions memory into required, admissible, and
prohibited sets. Prohibited memories are typed as superseded, deleted,
restricted, cross-namespace, or stale-tool.
Relevance is not enough: remembered evidence must also be… See the full description on the dataset page: https://huggingface.co/datasets/Multilingual-Multimodal-NLP/AutoMemoryBench.Multilingual-BenchmarkThese are the GSM8K and ARC dataset translated by Google Translate.
BibTex
@misc{lu2024languagecountslearnunlearn,
title={Every Language Counts: Learn and Unlearn in Multilingual LLMs},
author={Taiming Lu and Philipp Koehn},
year={2024},
eprint={2406.13748},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2406.13748},
}
kids-multilingual-benchmark
TinyAya v2 — Multilingual Benchmark for Children's AI Companions
2,312 child–AI conversational prompts across 23 languages, evaluated against
four models with five-judge LLM-as-judge validation.
📄 Companion article: see HF Articles by @batuhanaktas.
💻 Code: https://github.com/aktasbatuhan/cohere-tiny-aya-for-kids
Dataset summary
This dataset contains:
benchmark/items.jsonl — 2,312 benchmark items in 23 languages. Each item
is a structured prompt designed to mimic… See the full description on the dataset page: https://huggingface.co/datasets/batuhanaktas/kids-multilingual-benchmark.Balochi-Multilingual-dataset
Balochi Language Dataset
Overview
This dataset is a comprehensive resource for training large language models (LLMs) in the Balochi language. It is designed to go beyond basic translation tasks, supporting fully generative text and conversational AI capabilities in Balochi.
The dataset includes monolingual Balochi text, multilingual translation corpora, and various conversational and domain-specific texts, enabling diverse use cases such as:
Generative AI: Building… See the full description on the dataset page: https://huggingface.co/datasets/Salman95s/Balochi-Multilingual-dataset.kisan-advisory-multilingual-indic
Kisan Advisory (Hindi / Punjabi / English)
Real farmer questions and Farm Tele Advisor answers from India's government
Kisan Call Centre helpline, adapted with AutoScientist, expanded into
Hindi and Punjabi, and filtered so that every row provably preserves the
agrochemical doses in its source note.
Rows (after dose filtering)
6,232
Language split
2,064 en / 2,726 hi / 1,442 pa
Quality grade
E → C (3.0 → 5.8)
Relative improvement
+93.3%
Percentile
13.8… See the full description on the dataset page: https://huggingface.co/datasets/tojpaj/kisan-advisory-multilingual-indic.Balochi-Multilingual-dataset
Balochi Language Dataset
Overview
This dataset is a comprehensive resource for training large language models (LLMs) in the Balochi language. It is designed to go beyond basic translation tasks, supporting fully generative text and conversational AI capabilities in Balochi.
The dataset includes monolingual Balochi text, multilingual translation corpora, and various conversational and domain-specific texts, enabling diverse use cases such as:
Generative AI:… See the full description on the dataset page: https://huggingface.co/datasets/subhan87/Balochi-Multilingual-dataset.multilingual-addition
Multilingual Addition Dataset
Synthetic dataset of addition problems of the form a+b=answer, where a
and b are written-form representations of integers in 21 languages, plus
a 22nd split using raw digit strings.
Task format
Each sample contains:
field
type
description
a_str
str
written-form (or digit) representation of a
a_digit
int
integer value of a
b_str
str
written-form (or digit) representation of b
b_digit
int
integer value of b
answer
str… See the full description on the dataset page: https://huggingface.co/datasets/flexitok/multilingual-addition.Balochi-Multilingual-dataset
Balochi Language Dataset
Overview
This dataset is a comprehensive resource for training large language models (LLMs) in the Balochi language. It is designed to go beyond basic translation tasks, supporting fully generative text and conversational AI capabilities in Balochi.
The dataset includes monolingual Balochi text, multilingual translation corpora, and various conversational and domain-specific texts, enabling diverse use cases such as:
Generative AI: Building… See the full description on the dataset page: https://huggingface.co/datasets/shayak111/Balochi-Multilingual-dataset.indian-agri-advice-multilingual
This dataset is a quality-filtered golden subset prepared using a layered regex + local LLM-as-judge pipeline, then re-adapted using Adaption's Adaptive Data platform.
Indian Agricultural Advisory Dataset — Multilingual (Golden v5)
718 rows | 11 languages | 14 agro-climatic zones | 12 categories | 100% metadata fill rate
A multilingual agricultural advisory dataset covering 14 of India's 15 Planning Commission agro-climatic zones, localized to 11 Indian languages. This is a… See the full description on the dataset page: https://huggingface.co/datasets/vinod-anbalagan/indian-agri-advice-multilingual.IslamQA-Multilingual-Dataset
IslamQA-Multilingual-Dataset
Dataset Summary
IslamQA-Multilingual-Dataset is a comprehensive, multilingual dataset of Islamic rulings (Fatwas) scraped from the archive of IslamQA.info. It contains 67,059 Question-Answer pairs across 16 languages, structured in the standard ShareGPT conversational format.
This dataset was created to facilitate the training and evaluation of Large Language Models (LLMs) on Islamic jurisprudence (Fiqh) and general Islamic knowledge.… See the full description on the dataset page: https://huggingface.co/datasets/abdulmunimjemal/IslamQA-Multilingual-Dataset.Balochi-Multilingual-dataset
Balochi Language Dataset
Overview
This dataset is a comprehensive resource for training large language models (LLMs) in the Balochi language. It is designed to go beyond basic translation tasks, supporting fully generative text and conversational AI capabilities in Balochi.
The dataset includes monolingual Balochi text, multilingual translation corpora, and various conversational and domain-specific texts, enabling diverse use cases such as:
Generative AI: Building… See the full description on the dataset page: https://huggingface.co/datasets/mainkilora/Balochi-Multilingual-dataset.RAG_Multilingual
Dataset Card for RAG_Multilingual
Dataset Summary
RAG_Multilingual is an instruction-following synthetic QA dataset created from extractive QA datasets from Catalan, English and Spanish reference sets.
The reference datasets were: SQAD (https://huggingface.co/datasets/rajpurkar/squad), Catalanqa (https://huggingface.co/datasets/projecte-aina/catalanqa) and SQAC (https://huggingface.co/datasets/PlanTL-GOB-ES/SQAC).
This dataset, of 56.406 instances, was created by… See the full description on the dataset page: https://huggingface.co/datasets/OmLakshkar/RAG_Multilingual.agriculture-advisor-adapted-multilingual-v1
Agriculture Advisor (Adaption-adapted, multilingual) v1
The adapted dataset used to fine-tune our agricultural advisory, localised model for the
Adaption AutoScientist Challenge.
Produced by running 15juneee/agriculture-advisor-seed-v1 through
Adaption's datasets.run. The seed carries the prompts and the curation; this carries
the completions the model was actually trained on.
Rows
22,270 rows. Adaption writes its output to enhanced_prompt / enhanced_completion… See the full description on the dataset page: https://huggingface.co/datasets/15juneee/agriculture-advisor-adapted-multilingual-v1.indian-agri-advice-multilingual
This dataset is a quality-filtered golden subset prepared using a layered regex + local LLM-as-judge pipeline, then re-adapted using Adaption's Adaptive Data platform.
Indian Agricultural Advisory Dataset — Multilingual (Golden v5)
718 rows | 11 languages | 14 agro-climatic zones | 12 categories | 100% metadata fill rate
A multilingual agricultural advisory dataset covering 14 of India's 15 Planning Commission agro-climatic zones, localized to 11 Indian languages. This is a… See the full description on the dataset page: https://huggingface.co/datasets/prasannachinna2006/indian-agri-advice-multilingual.Multilingual-Nepali-Customer-Care-Services-Datasetclean_multilingual_thinking元データ: https://huggingface.co/datasets/HuggingFaceH4/Multilingual-Thinking
使用したコード: https://github.com/LLMTeamAkiyama/0-data_prepare/tree/master/src/Multilingual-Thinking
データ件数: 197
平均トークン数: 872
最大トークン数: 2,339
合計トークン数: 171,812
ファイル形式: JSONL
ファイル分割数: 1
合計ファイルサイズ: 748.5 KB
加工内容：
フィルタリングによるデータクレンジング
言語フィルタリング: reasoning_languageが「English」のデータのみを抽出します。
文字数フィルタリング: 処理速度の観点から、question（質問）、thought（思考）、answer（回答）の各フィールドで、規定の文字数を超える長大なデータは事前に除外します。
繰り返し表現の除去:… See the full description on the dataset page: https://huggingface.co/datasets/LLMTeamAkiyama/clean_multilingual_thinking.multilingual-repo-qa
UPD: Release in progress, stay tuned.
MultiRepoQA
MultiRepoQA is a multilingual benchmark for question answering over complete software repositories. It contains 783 validated canonical questions across 30 open-source repositories, with aligned English, German, and Ukrainian versions (2,349 language-specific examples). Questions cover local implementation details, cross-file behavior, repository-wide flows, and maintenance impact.
The benchmark was introduced in… See the full description on the dataset page: https://huggingface.co/datasets/NLPForUA/multilingual-repo-qa.multilingual-rag-finance-law
Multilingual RAG Benchmark: Hindi, Italian & Japanese Finance and Law
A BEIR-compatible retrieval and end-to-end RAG evaluation benchmark for
Hindi (hi), Italian (it) and Japanese (ja) across two specialist domains,
finance and law. Use it to benchmark multilingual text embedding models
and dense retrievers (NDCG@10 with TREC-style qrels), and to measure
whether retrieval-quality differences actually propagate to downstream
answer quality — in languages that aggregated… See the full description on the dataset page: https://huggingface.co/datasets/ArchitRastogi/multilingual-rag-finance-law.