datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Fact-Completion
Dataset Card
Homepage: https://bit.ly/ischool-berkeley-capstone
Repository: https://github.com/daniel-furman/Capstone
Point of Contact: daniel_furman@berkeley.edu
Dataset Summary
This is the dataset for Polyglot or Not?: Measuring Multilingual Encyclopedic Knowledge Retrieval from Foundation Language Models.
Test Description
Given a factual association such as The capital of France is Paris, we determine whether a model adequately "knows" this… See the full description on the dataset page: https://huggingface.co/datasets/Polyglot-or-Not/Fact-Completion.Polyglot-Thoughts-SFT-Collection
Polyglot Thoughts SFT Collection
Polyglot Thoughts SFT Collection is a large-scale supervised fine-tuning (SFT)
corpus for reasoning-oriented language models. It combines, filters, deduplicates,
and language-extends a broad set of public reasoning datasets into a single uniform
schema centred on chain-of-thought reasoning traces.
The final corpus contains 23,896,757 examples and roughly 123 billion tokens,
spanning six languages (English, German, French, Italian, Spanish… See the full description on the dataset page: https://huggingface.co/datasets/ahmad21omar/Polyglot-Thoughts-SFT-Collection.polyglot-ko-tokenizer-corpus
Dataset Card for "polyglot-ko-tokenizer-corpus"
More Information needed
polyglotoxicitypromptswikipedia-language-snippets-filtered
Wikipedia Snippets (Filtered)
Filtered sentence snippets in Wikipedia, by taking the first 60% of an article after filtering for stubs. Minor Latin groups are additionally filtered again for English leakage.
Sentences are mostly filtered out for non matching scripts, such as Arabic in a Cyrllic language.
Files
Each file is in this format for languages in ISO 639 2-letter codes:
train/en/en.parquet
train/es/es.parquet
From wikimedia/wikipedia
Licensing… See the full description on the dataset page: https://huggingface.co/datasets/polyglot-tagger/wikipedia-language-snippets-filtered.finetranslations-filtered
Files
Each file is in this format for languages in ISO 639 2-letter codes:
train/en/en.parquet
train/es/es.parquet
From HuggingFaceFW/finetranslations
Licensing Information
The dataset is released under the Open Data Commons Attribution License (ODC-By) v1.0 license. The use of this dataset is also subject to CommonCrawl's Terms of Use.
Citation Information
@misc{penedo2026finetranslations,
title={FineTranslations},
author={Guilherme… See the full description on the dataset page: https://huggingface.co/datasets/polyglot-tagger/finetranslations-filtered.Polyglot-Thoughts-RL-Collection
Polyglot Thoughts RL Collection
Polyglot Thoughts RL Collection is a large-scale, curated corpus for
reinforcement learning from verifiable rewards (RLVR) of reasoning-oriented
language models. It combines, filters, normalises, and deduplicates a broad set
of public RL datasets into a single uniform schema in which every row carries a
machine-verifiable ground-truth signal — math equivalence, code execution,
Prolog rule induction, schema validation, multiple-choice… See the full description on the dataset page: https://huggingface.co/datasets/ahmad21omar/Polyglot-Thoughts-RL-Collection.PolyglotTeachers-SFT-Synth-Data
Website: ljvmiranda921.github.io/polyglot-teachers/
PolyglotTeachers-SFT-Synth
This dataset contains synthetic supervised fine-tuning examples generated by the best teacher we found in the paper Polyglot Teachers: Evaluating Language Models for Multilingual Synthetic Data Generation, where we systematically characterize what makes a good teacher model.
It contains examples across six languages: Arabic, Czech, German, Indonesian, Japanese, Spanish, and Tagalog. Note: In… See the full description on the dataset page: https://huggingface.co/datasets/ljvmiranda921/PolyglotTeachers-SFT-Synth-Data.tatoeba-filtered
Files
Each file is in this format for languages in ISO 639 2-letter codes:
train/en/en.parquet
train/es/es.parquet
aider_polyglot_a3_rl_DCAgent_inferredbugs_sandboxes_verifier_55_8B_20260526_230125aider_polyglot_GLM_4_7_swesmith_sandboxes_with_tests_oracle_verified_120s_maxepf48469bfaider_polyglot_a3_rl_DCAgent_selfinstruct_naive_sandboxes_2_verified_70_8B_202690541bcechad-polyglot-runs
chad run data
Run output from the benchmarks of chad, a local
coding agent for Apple Silicon, kept out of the code repository. Every polyglot run here is
a row of benchmarks/polyglot/RUNS.md
with the sha256 of its trials.jsonl, and benchmarks/polyglot/fetch.py --label <label>
downloads one and refuses it if the hash differs.
path
what
polyglot/<label>/
one polyglot run: meta.json (chad version and commit, model, context limit, every CHAD_* variable), trials.jsonl (one… See the full description on the dataset page: https://huggingface.co/datasets/nathansutton/chad-polyglot-runs.MADLAD_CulturaX_cleanedPolyglotAudio
Citation
If you use this dataset in your research or downstream work, please cite:
@misc{polyglot_audio_2026,
author = {Fernandes, Reuben Chagas},
title = {PolyglotAudio: Multilingual Audio Pre-training Corpus},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/datasets/Reubencf/PolyglotAudio}}
}
APA-style:
Reuben Chagas Fernandes (2026). PolyglotAudio: Multilingual Audio Pre-training Corpus [Dataset]. Hugging… See the full description on the dataset page: https://huggingface.co/datasets/Reubencf/PolyglotAudio.DCAgent2_aider_polyglot_DCAgent2_swesmith-stack-reason_20260127_005703nlp-noise-snippets
Synthetic Noise Pool
For Text Classification purposes, as many models may consider code snippets, html artifacts, and math as "English".
Around 50K are latex snippets from im2latex-100k
DCAgent2_aider_polyglot_DCAgent2_nl2bash-swesmith-reason_20260127_005703polyglot_paylods_datasets
Polyglot Payloads Dataset for Cybersecurity Training
Overview
This dataset, polyglot_payloads.jsonl, is a curated collection of 500 polyglot payloads designed for training AI models in cybersecurity, specifically for red team operations and vulnerability detection. The dataset includes payloads targeting common web vulnerabilities such as Cross-Site Scripting (XSS), SQL Injection (SQLi), Local File Inclusion (LFI), Remote Code Execution (RCE), and Server-Side Template… See the full description on the dataset page: https://huggingface.co/datasets/darkknight25/polyglot_paylods_datasets.DCAgent2_aider_polyglot_DCAgent_freelancer-long-instruction-filter_Qwen3-8B_2021a649aaeDCAgent2_aider_polyglot_DCAgent_tbench_oracle_solutions_terminus_20260125_202032PolygloToxicityPrompts_permissive
Polyglo PTP Permissive
Permissive-source subset of the PTP configs from ToxicityPrompts/PolygloToxicityPrompts, filtered using the local URL/source permissiveness rules in this workspace. WildChat configs are not included because they do not provide row-level source URLs.
Source split: full for ptp-* configs.
Rows scanned: 425,000. Rows kept: 1,649.
See polyglo_ptp_permissive_stats_620972.json for kept/discard reason distributions and per-language counts.
polyglot-modelsThis dataset repository shall serve as a mirror hosting models for polyglot.
Availability
Please note that currently the only languages, which have all models, are English (en) and Bulgarian (bg). Other languages may have partial support.
Adding missing models
In case you have previously downloaded polyglot language models, which are not available in this repo, please open a Pull Request.
License
All rights belong to the original authors. Please refer to the… See the full description on the dataset page: https://huggingface.co/datasets/ndandanov/polyglot-models.DCAgent2_aider_polyglot_DCAgent_tbench-dev-71-nl2bash-bugsseq_Qwen3-8B-8nodes-sfd6ea99fDCAgent2_aider_polyglot_DCAgent2_test2-tbench-dev-71-qwen3-8b-8nodes-sync_2026087639023polyglot-ko-tokenizer-corpus-merge_ws
Dataset Card for "polyglot-ko-tokenizer-corpus-merge_ws"
More Information needed
polyglot_fr_en_subset
Dataset Card for "polyglot_fr_en_subset"
More Information needed
DCAgent2_aider_polyglot_laion_glm46-qasper-maxeps-131k_20260121_072458aider_polyglot_Qwen3_30B_A3B_Instruct_2507_20260425_063500-tracesDCAgent2_aider_polyglot_DCAgent_freelancer-askllm-filtered-sandboxes-traces-terf2137443