datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
fineweb-2-dutchbagaco3
Bagaço3 🍷🇵🇹
Bagaço3 is the third version of Bagaço, the largest pretraining dataset for European Portuguese. It follows Bagaço2 and adds documents from FinePDFs and FineWiki.
Bagaço collects European Portuguese documents from upstream sources and adds an educational score and content category to each document. See Classification for details.
Methodology
Collect documents from Bagaço2, FinePDFs, and FineWiki.
Filter new FinePDFs and FineWiki documents with the… See the full description on the dataset page: https://huggingface.co/datasets/duarteocarmo/bagaco3.fineweb2-bagaco
Bagaço 🍷🇵🇹
Bagaço is a pretraining dataset for European Portuguese. It filters the Fineweb2 dataset to URLs from Portuguese domains (e.g., .pt/). Each document is classified into one of 9 categories and scored for educational quality.
Filtering
Source: HuggingFaceFW/fineweb-2, subset por_Latn, split train
Filter: URLs containing .pt/ (Portuguese top-level domain)
Document classification
Each document is classified into one of 9 categories: Society, Arts… See the full description on the dataset page: https://huggingface.co/datasets/duarteocarmo/fineweb2-bagaco.DutchMedicalTextV3Translated mix of PMC/Pubmed, existing data sets from Apollo/Meditron, various Dutch sources and now with
the medically filtered texts from fineweb and finepdf.
Be aware of spurious repetitions due to the use of neural machine translation. You can mitigate it partially by iteratively applying a suitable regex-replace
operation. Example (bleeding eyes) code is
RE_SPURIOUS_CHARS = re.compile(r'([^\w])\1{3,}')
RE_SPURIOUS_WORDS = re.compile(r'(\b[\w\-\s\;\:\,\.]+\b)\1{4,}')
RE_MULTISPACE =… See the full description on the dataset page: https://huggingface.co/datasets/UMCU/DutchMedicalTextV3.DutchMedicalTextV2Translated mix of PMC/Pubmed, existing data sets from Apollo/Meditron and various Dutch sources.
Be aware of spurious repetitions due to the use of neural machine translation. You can mitigate it partially by iteratively applying a suitable regex-replace
operation. Example (bleeding eyes) code is
RE_SPURIOUS_CHARS = re.compile(r'([^\w])\1{3,}')
RE_SPURIOUS_WORDS = re.compile(r'(\b[\w\-\s\;\:\,\.]+\b)\1{4,}')
RE_MULTISPACE = re.compile(r'\s{2,}')
def apply_until_stable(pattern, repl, text… See the full description on the dataset page: https://huggingface.co/datasets/UMCU/DutchMedicalTextV2.PubmedAbstracts_Dutch_v1
License:
All Rights Reserved with Limited ML Training License.
This data may only be used for training machine-learning models and may not be redistributed in its original or substantially similar form.
This refers explicitly to sharing the data as-is, or as-derived. It does not prohibit you to use this data for training machine learning models as this would constitute FAIR use depending on the application.
Articles available from PubMed Central (PMC) are provided by the respective… See the full description on the dataset page: https://huggingface.co/datasets/UMCU/PubmedAbstracts_Dutch_v1.pierogue
Pierogue
Pierogue is a small open-licensed machine-generated dataset that contains fifteen short texts in English covering five topics, provided with the relevance judgements (qrels), designed for educational purposes.
Topics: cosmos, nature, music, technology, fashion
Splits: train (10 documents, 375 qrels) and test (5 documents, 150 qrels)
Texts were generated by ChatGPT 3.5. Queries, qrels, and analogies were generated by GPT-4. Words were provided with Word2Vec embeddings… See the full description on the dataset page: https://huggingface.co/datasets/dustalov/pierogue.ultra_feedback_dutch_cleaned
Ultra Feedback Dutch Cleaned
This is a cleaned version of BramVanroy/ultra_feedback_dutch, based on the cleaning done by Argilla on the original Ultra Feedback dataset. Another difference is that we only include GEITje 7B Ultra and GPT-4-Turbo. GEITje chat, which was used in the original dataset, is not used.
After cleaning I also generated replies for other models (like TowerInstruct, Mistral), but the results were too poor (in Dutch) to include so we only kept the GEITje Ultra and… See the full description on the dataset page: https://huggingface.co/datasets/BramVanroy/ultra_feedback_dutch_cleaned.turkish-court-decisions-duplicate
Türk İçtihat Korpusu — 11.045.085 Mahkeme Kararı
Türkiye'nin kamuya açık mahkeme kararlarından derlenmiş, bilinen en büyük Türkçe
hukuk metni veri seti. 11.045.085 karar, 31.5 milyar karakter düz metin (5.50 GB Parquet),
1962'den 2026'ya. Yargıtay, Danıştay, Anayasa Mahkemesi ve UYAP Emsal üzerinden
yerel/istinaf mahkemeleri.
Kapsam
Kaynak
Karar sayısı
Yıl aralığı
Metin
Dosya
Yargıtay (yargitay)
9.820.145
1997–2026
19.5 milyar karakter
17
Danıştay… See the full description on the dataset page: https://huggingface.co/datasets/serdarsrts/turkish-court-decisions-duplicate.DualBlind
GlimmaryKarl/DualBlind
Curated Frontier Reasoning and Direct Preference Optimization (DPO) Dataset Generated from Double-Blind Multi-Agent Arena Evaluations.
This dataset was generated using the DualBlind AI Benchmark Arena. In this setup, two independent frontier AI models engage in multi-turn double-blind dialogue to solve extreme-difficulty benchmark problems, verifying their peer's proofs, raising counter-examples, and reaching mathematical consensus.
Dataset… See the full description on the dataset page: https://huggingface.co/datasets/GlimmaryKarl/DualBlind.duplex-qa-refusal
duplex-qa-refusal
No dialogue in this set has been validated by a human.
Text-side augmentation of the moshika spoken-QA corpus so a full-duplex speech model can be trained to refuse a query when a mid-conversation text instruction tells it to, voice the reason the instruction gives, and then carry on normally. Two classes: policy (an existing benign query is declined for a stated reason; comes with an untouched accept twin sharing pair_id) and attack (a new user turn pivots to… See the full description on the dataset page: https://huggingface.co/datasets/MagicLuke/duplex-qa-refusal.l4-gpu-llm-benchmark-leaderboard
🚀 Local LLM Serving & Quality Benchmark Leaderboard (NVIDIA L4 24GB)
An exhaustive, reproducible benchmark study measuring real-world serving performance (TTFT, TPOT, throughput, peak VRAM, energy consumption, and cost) alongside rigorous task quality gates (HumanEval+, MMLU-Pro, BFCL v4 tool calling, and RULER needle retrieval) for open-weight LLMs on a single NVIDIA L4 24GB GPU.
📊 Executive Summary & Key Takeaways
⚡ Best Throughput & Coding Workhorse:… See the full description on the dataset page: https://huggingface.co/datasets/mayank-dubey-ai/l4-gpu-llm-benchmark-leaderboard.forecastbench-single_question
ForecastBench Single Questions
This dataset contains single-ID forecasting questions derived from the ForecastBench project. It includes two configurations:
forecastbench_single_questions_2024-12-08: Contains 429 forecasting questions with resolved real-world outcomes.
forecastbench_single_questions_human_2024-07-21: Contains 473 questions with resolved real-world outcomes, augmented with human forecast probabilities from public and superforecaster groups.
Dataset… See the full description on the dataset page: https://huggingface.co/datasets/Duruo/forecastbench-single_question.orca_dpo_pairs_dutch_cleaned
Dataset Card for Orca DPO Pairs Dutch Cleaned
Citation
If you use this dataset, GEITje 7B Ultra (SFT) or any of its derivatives or quantizations, place cite the following paper:
@misc{vanroy2024geitje7bultraconversational,
title={GEITje 7B Ultra: A Conversational Model for Dutch},
author={Bram Vanroy},
year={2024},
eprint={2412.04092},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2412.04092},
}… See the full description on the dataset page: https://huggingface.co/datasets/BramVanroy/orca_dpo_pairs_dutch_cleaned.llmfao
Large Language Model Feedback Analysis and Optimization (LLMFAO)
The original Crowdsourced LLM Benchmark dataset in files prompts.parqet and outputs.parquet was kindly provided by the team at llmonitor.com under a CC BY 4.0 license. This dataset can be conveniently processed with Evalica (arXiv).
common_corpus_dutch_pd
Common Corpus v2 - Dutch Public Domain collection
This is a version of Common Corpus v2 filtered to keep only the rows where collection is "Dutch-PD".
Looking for all Dutch-language documents in Common Corpus, regardless of the collection they are in?
Then you might want to look at Rijgersberg/common_corpus_nl.
Common Corpus is a very large open and permissible licensed text dataset created by Pleias.
Please be sure to acknowledge the creators of the original dataset when using this… See the full description on the dataset page: https://huggingface.co/datasets/Rijgersberg/common_corpus_dutch_pd.ClawBench-test
ClawBench
Can AI Agents Complete Everyday Online Tasks?
ClawBench evaluates AI agents on 153 everyday tasks (such as booking flights, ordering groceries, submitting job applications) across 144 live websites. We capture 5 layers of behavioral data (session replay, screenshots, HTTP traffic, agent reasoning traces, and browser actions), collect human ground-truth for every task, and score with an agentic evaluator that provides step-level traceable diagnostics.
Paper… See the full description on the dataset page: https://huggingface.co/datasets/Duke313/ClawBench-test.SimpleStories
📘📕 SimpleStories 📙📗
SimpleStories is a dataset of >2 million model-generated short stories. It was made to train small, interpretable language models on it. The generation process is open-source: To see how the dataset was generated, or to generate some stories yourself, head over to this repository.
If you'd like to commission other languages or story formats, feel free to send mail.
When using SimpleStories in your work, please cite the SimpleStories paper:… See the full description on the dataset page: https://huggingface.co/datasets/duoduoyeah/SimpleStories.lalm-judge-validation-full-duplex
LALM Judge Validation on Full-Duplex Voice Agents
Companion dataset for the paper A Reliability Assessment of
LALM Audio Judges for Full-Duplex Voice Agents.
This repository contains the anonymised ratings, adversarial-defect
recall tables, JSON schemas, and analysis scripts used to produce
every headline number, table, and figure in that paper.
Summary
209 rated stereo sessions: 152 full-duplex agent-client
conversations across 13 accent-and-condition strata… See the full description on the dataset page: https://huggingface.co/datasets/Salesforce/lalm-judge-validation-full-duplex.DuET-PD
DuET-PD: Dual Evaluation for Trust in Persuasive Dialogues
Dataset Summary
DuET-PD is a comprehensive framework and dataset designed to evaluate the robustness and adaptability of Large Language Models (LLMs) in multi-turn persuasive dialogues. The dataset probes an LLM's ability to navigate the critical tension between resisting misinformation (robustness) and accepting valid corrections (adaptability).
The "Dual" aspect of DuET-PD reflects its two core evaluation… See the full description on the dataset page: https://huggingface.co/datasets/Incomple/DuET-PD.dual-diagnosis-dataset
دیتاست پروتکل تشخیص دوگانه (فارسی)
پایگاه دانش و دادهی آموزشِ دستیار بالینی RAG برای تشخیص دوگانه
(سایکوز + اعتیاد + BPD ± ADHD) — مبتنی بر NICE · APA · WFSBP.
فایلها
protocol.md — پایگاه دانش پروتکل (۴۵ قطعه).
instruction_pairs.jsonl — جفتهای پرسشوپاسخ برای fine-tune.
index/chunks.json + index/vectors.npz — ایندکس برداری از پیش ساختهشده (امبدینگ چندزبانه MiniLM، ۳۸۴ بُعد).
نحوهی استفاده
from datasets import load_dataset
ds =… See the full description on the dataset page: https://huggingface.co/datasets/sosa123454321/dual-diagnosis-dataset.gemma4-qwen35-gsm8k-rollouts
Gemma 4 and Qwen3.5 GSM8K Rollouts
This dataset contains 3,957 saved generations from three complete runs over
the 1,319-example openai/gsm8k main test split:
Model
Rows
Strict match
Flexible extract
google/gemma-4-26B-A4B
1,319
33.28%
39.95%
google/gemma-4-E4B
1,319
26.23%
30.86%
Qwen/Qwen3.5-35B-A3B
1,319
15.92%
23.12%
Every row includes the exact five-shot prompt, model generation, reference
answer, strict and flexible correctness flags, pinned… See the full description on the dataset page: https://huggingface.co/datasets/dureduck/gemma4-qwen35-gsm8k-rollouts.dumy-zno-ukrainian-math-history-geo-r1-o1
DUMY («Думи»): Ukrainian Multidomain Reasoning Dataset (Part 1: ZNO/NMT tasks with DeepSeek R1 and OpenAI o1 answers)
DUMY is an open benchmark and dataset designed for training, distillation, and evaluation of language models focused on Ukrainian reasoning tasks.
The word “Dumy” comes from Taras Shevchenko’s famous poem and literally means “thoughts” in Ukrainian:
Думи мої, думи мої,
Лихо мені з вами!
Нащо стали на папері
Сумними рядами?..
Work in progress. Stay tuned.… See the full description on the dataset page: https://huggingface.co/datasets/NLPForUA/dumy-zno-ukrainian-math-history-geo-r1-o1.kitrec-dualft_music-setb
KitREC DUALFT_MUSIC - Set B
DualFT model for Music recommendations with overlapping and cold-start users
Dataset Description
This dataset is part of the KitREC (Knowledge-Instruction Transfer for Recommendation) research project, designed for fine-tuning LLMs on cross-domain recommendation tasks.
Dataset Summary
Attribute
Value
Model Type
dualft_music
Candidate Set
Set B (Random (Fair baseline))
Target Domain
Music
Source Domain
Books
Total… See the full description on the dataset page: https://huggingface.co/datasets/Younggooo/kitrec-dualft_music-setb.apollo_english_guidelines_translated_to_dutch_with_nllb200
Data description
Translation of the English medical guidelines that are part of the Apollo corpus, using the NLLB200-600M NTM.
Acknowledgement
The work received funding from the European Union's Horizon Europe research
and innovation programme under Grant Agreement No. 101057849 (DataTools4Heart project).
For more information on the background, see Datatools4Heart Huggingface/Website/Git
apollo_english_guidelines_translated_to_dutch_with_marianmt
Data description
Apollo corpus, English guidelines translated to Dutch using MariaNMT.
Acknowledgement
The work received funding from the European Union's Horizon Europe research
and innovation programme under Grant Agreement No. 101057849 (DataTools4Heart project).
For more information on the background, see Datatools4Heart Huggingface/Website/Git
kitrec-dualft_music-seta
KitREC DUALFT_MUSIC - Set A
DualFT model for Music recommendations with overlapping and cold-start users
Dataset Description
This dataset is part of the KitREC (Knowledge-Instruction Transfer for Recommendation) research project, designed for fine-tuning LLMs on cross-domain recommendation tasks.
Dataset Summary
Attribute
Value
Model Type
dualft_music
Candidate Set
Set A (Hybrid (Hard negatives + Random))
Target Domain
Music
Source Domain
Books… See the full description on the dataset page: https://huggingface.co/datasets/Younggooo/kitrec-dualft_music-seta.biolord_dutch_marianmt
Dataset Card for Biolord Dutch Marianmt
MariaNMT translation of BioLORD definitions.
The source language: English
The original data source: Original Data Source
Data description
Dutch translation of BioLord definitions using MariaMT
Acknowledgement
This is part of the DT4H project with attribution [Cite the paper].
Doi and reference
[DOI and reference to the source paper/s]
kitrec-dualft_movies-seta
KitREC DUALFT_MOVIES - Set A
DualFT model for Movies & TV recommendations with overlapping and cold-start users
Dataset Description
This dataset is part of the KitREC (Knowledge-Instruction Transfer for Recommendation) research project, designed for fine-tuning LLMs on cross-domain recommendation tasks.
Dataset Summary
Attribute
Value
Model Type
dualft_movies
Candidate Set
Set A (Hybrid (Hard negatives + Random))
Target Domain
Movies & TV… See the full description on the dataset page: https://huggingface.co/datasets/Younggooo/kitrec-dualft_movies-seta.epfl_guidelines_dutch_marianmt
Dataset Card for Epfl English Guidelines Translated To Dutch With MariaNMT
This dataset was created by the EPFL, and can found in it original form here
The source language: English
The original data source: Original Data Source
The MariaNMT model used can be found: here
Data description
Translation of the English medical guidelines that are part of the Meditron corpus, using the LLM GPT 4o mini
Acknowledgement
This is part of the DT4H project with… See the full description on the dataset page: https://huggingface.co/datasets/UMCU/epfl_guidelines_dutch_marianmt.