datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
wikipedia_culturax_dutch
Filtered CulturaX + Wikipedia for Dutch
This is a combined and filtered version of CulturaX and Wikipedia, only including Dutch. It is intended for the training of LLMs.
Different configs are available based on the number of tokens (see a section below with an overview). This can be useful if you want to know exactly how many tokens you have. Great for using as a streaming dataset, too. Tokens are counted as white-space tokens, so depending on your tokenizer, you'll likely end up… See the full description on the dataset page: https://huggingface.co/datasets/BramVanroy/wikipedia_culturax_dutch.fineweb-2-dutchdutch-dynaword
🧨 Dutch Dynaword
Version
1.0.1 (Changelog)
Language
nld, Nederlands, Dutch
License
Openly Licensed, See the respective dataset
Models
For model trained used this data see danish-foundation-models
Contact
If you have question about this project please create an issue here
Dataset Description
Number of samples: 14.45M
Number of tokens (Llama 3): 37.89B
Average document length in tokens (min, max): 2.62K (2, 5.45M)
Dataset… See the full description on the dataset page: https://huggingface.co/datasets/danish-foundation-models/dutch-dynaword.DutchMedicalTextV3Translated mix of PMC/Pubmed, existing data sets from Apollo/Meditron, various Dutch sources and now with
the medically filtered texts from fineweb and finepdf.
Be aware of spurious repetitions due to the use of neural machine translation. You can mitigate it partially by iteratively applying a suitable regex-replace
operation. Example (bleeding eyes) code is
RE_SPURIOUS_CHARS = re.compile(r'([^\w])\1{3,}')
RE_SPURIOUS_WORDS = re.compile(r'(\b[\w\-\s\;\:\,\.]+\b)\1{4,}')
RE_MULTISPACE =… See the full description on the dataset page: https://huggingface.co/datasets/UMCU/DutchMedicalTextV3.DutchMedicalTextV2Translated mix of PMC/Pubmed, existing data sets from Apollo/Meditron and various Dutch sources.
Be aware of spurious repetitions due to the use of neural machine translation. You can mitigate it partially by iteratively applying a suitable regex-replace
operation. Example (bleeding eyes) code is
RE_SPURIOUS_CHARS = re.compile(r'([^\w])\1{3,}')
RE_SPURIOUS_WORDS = re.compile(r'(\b[\w\-\s\;\:\,\.]+\b)\1{4,}')
RE_MULTISPACE = re.compile(r'\s{2,}')
def apply_until_stable(pattern, repl, text… See the full description on the dataset page: https://huggingface.co/datasets/UMCU/DutchMedicalTextV2.dutch-corpus-200b
Dutch Corpus 200B (DC-200B)
Dataset Summary
The Dutch Corpus 200B (DC-200B) is the largest open-source, deduplicated, and professionally cleaned dataset designed for training Foundation Models in the Dutch language. Comprising approximately 202 Billion tokens (measured with Qwen 2.5 tokenizer), it bridges the gap between high-resource English models and the Dutch ecosystem.
The dataset is engineered for a two-stage training pipeline:
Pretrain Subset (~195B… See the full description on the dataset page: https://huggingface.co/datasets/hasankursun/dutch-corpus-200b.ultra_feedback_dutch_cleaned
Ultra Feedback Dutch Cleaned
This is a cleaned version of BramVanroy/ultra_feedback_dutch, based on the cleaning done by Argilla on the original Ultra Feedback dataset. Another difference is that we only include GEITje 7B Ultra and GPT-4-Turbo. GEITje chat, which was used in the original dataset, is not used.
After cleaning I also generated replies for other models (like TowerInstruct, Mistral), but the results were too poor (in Dutch) to include so we only kept the GEITje Ultra and… See the full description on the dataset page: https://huggingface.co/datasets/BramVanroy/ultra_feedback_dutch_cleaned.PubmedAbstracts_Dutch_v1
License:
All Rights Reserved with Limited ML Training License.
This data may only be used for training machine-learning models and may not be redistributed in its original or substantially similar form.
This refers explicitly to sharing the data as-is, or as-derived. It does not prohibit you to use this data for training machine learning models as this would constitute FAIR use depending on the application.
Articles available from PubMed Central (PMC) are provided by the respective… See the full description on the dataset page: https://huggingface.co/datasets/UMCU/PubmedAbstracts_Dutch_v1.fineweb-dutch-edu-mt
FineWeb-Edu Dutch Machine Translated
Machine-translated Dutch text dataset derived from the FineWeb-Edu corpus.
Dataset Details
Source: HuggingFaceFW/fineweb-edu (sample-10BT subset)
Translation: English → Dutch using Unbabel/Tower-Plus-9B
Size: Up to 1.5M samples
Format: Translated text with original metadata
Schema
text: Machine-translated Dutch text
id: Original sample identifier from FineWeb-Edu
url: Source URL
Quality Notice
⚠️ This… See the full description on the dataset page: https://huggingface.co/datasets/pdelobelle/fineweb-dutch-edu-mt.ultrachat_200k_dutch
Dataset Card for UltraChat 200k Dutch
Citation
If you use this dataset, GEITje 7B Ultra (SFT) or any of its derivatives or quantizations, place cite the following paper:
@misc{vanroy2024geitje7bultraconversational,
title={GEITje 7B Ultra: A Conversational Model for Dutch},
author={Bram Vanroy},
year={2024},
eprint={2412.04092},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2412.04092},
}… See the full description on the dataset page: https://huggingface.co/datasets/BramVanroy/ultrachat_200k_dutch.stackoverflow-chat-dutch
Dataset Card for Stack Overflow Chat Dutch
Dataset Summary
This dataset contains 56,964 conversations between een AI assistant and a (fake) "Human" (generated) in Dutch, specifically in the domain of programming (Stack Overflow). They are translations of Baize's machine-generated answers to the Stack Overflow dataset.
☕ Want to help me out? Translating the data with the OpenAI API, and prompt testing, cost me 💸$133.60💸. If you like this dataset, please consider buying… See the full description on the dataset page: https://huggingface.co/datasets/BramVanroy/stackoverflow-chat-dutch.orca_dpo_pairs_dutch_cleaned
Dataset Card for Orca DPO Pairs Dutch Cleaned
Citation
If you use this dataset, GEITje 7B Ultra (SFT) or any of its derivatives or quantizations, place cite the following paper:
@misc{vanroy2024geitje7bultraconversational,
title={GEITje 7B Ultra: A Conversational Model for Dutch},
author={Bram Vanroy},
year={2024},
eprint={2412.04092},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2412.04092},
}… See the full description on the dataset page: https://huggingface.co/datasets/BramVanroy/orca_dpo_pairs_dutch_cleaned.quora-chat-dutch
Dataset Card for Quora Chat Dutch
Dataset Summary
Update 15/1/2024: the format of this dataset has been updated so that it now follows the messages format, where each conversation is a list of messages (dictionaries) and each dictionary has a role (user, assistant or system) and content key. Importantly, this dataset was also filtered by its language identification tag. We ran LID with FastText (results are in the separate 3-lid branch) and filtered on Dutch only.… See the full description on the dataset page: https://huggingface.co/datasets/BramVanroy/quora-chat-dutch.no_robots_dutch
Dataset Card for No Robots Dutch
Citation
If you use this dataset, GEITje 7B Ultra (SFT) or any of its derivatives or quantizations, place cite the following paper:
@misc{vanroy2024geitje7bultraconversational,
title={GEITje 7B Ultra: A Conversational Model for Dutch},
author={Bram Vanroy},
year={2024},
eprint={2412.04092},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2412.04092},
}
Dataset… See the full description on the dataset page: https://huggingface.co/datasets/BramVanroy/no_robots_dutch.common_corpus_dutch_pd
Common Corpus v2 - Dutch Public Domain collection
This is a version of Common Corpus v2 filtered to keep only the rows where collection is "Dutch-PD".
Looking for all Dutch-language documents in Common Corpus, regardless of the collection they are in?
Then you might want to look at Rijgersberg/common_corpus_nl.
Common Corpus is a very large open and permissible licensed text dataset created by Pleias.
Please be sure to acknowledge the creators of the original dataset when using this… See the full description on the dataset page: https://huggingface.co/datasets/Rijgersberg/common_corpus_dutch_pd.orca_dpo_pairs_dutch
Dataset Card for Orca DPO Pairs Dutch
[!TIP]
I recommend using the cleaned, deduplicated version. https://huggingface.co/datasets/BramVanroy/orca_dpo_pairs_dutch_cleaned
Citation
If you use this dataset, GEITje 7B Ultra (SFT) or any of its derivatives or quantizations, place cite the following paper:
@misc{vanroy2024geitje7bultraconversational,
title={GEITje 7B Ultra: A Conversational Model for Dutch},
author={Bram Vanroy},
year={2024}… See the full description on the dataset page: https://huggingface.co/datasets/BramVanroy/orca_dpo_pairs_dutch.alpaca-cleaned-dutch
Dataset Card for Alpaca Cleaned Dutch
Dataset Summary
This dataset contains 51,712 conversations between een AI assistant and a (fake) "Human" (generated) in Dutch. They are translations of Alpaca Cleaned Dataset.
☕ Want to help me out? Translating the data with the OpenAI API, and prompt testing, cost me 💸$57.99💸. If you like this dataset, please consider buying me a coffee to offset a portion of this cost, I appreciate it a lot! ☕
If you use this dataset or refer to… See the full description on the dataset page: https://huggingface.co/datasets/BramVanroy/alpaca-cleaned-dutch.ultra_feedback_dutch
Dataset Card for Ultra Feedback Dutch
[!WARNING]
It is recommended to use the cleaned version for your experiments.
Citation
If you use this dataset, GEITje 7B Ultra (SFT) or any of its derivatives or quantizations, place cite the following paper:
@misc{vanroy2024geitje7bultraconversational,
title={GEITje 7B Ultra: A Conversational Model for Dutch},
author={Bram Vanroy},
year={2024},
eprint={2412.04092},
archivePrefix={arXiv}… See the full description on the dataset page: https://huggingface.co/datasets/BramVanroy/ultra_feedback_dutch.nemotron-dutch-mt
Nemotron Post-Training Dataset (Dutch Translation)
Machine-translated Dutch version of NVIDIA's Nemotron Post-Training Dataset, specifically the chat conversations.
Dataset Details
Source: nvidia/Nemotron-Post-Training-Dataset-v2 (chat split)
Translation: English → Dutch using Unbabel/Tower-Plus-9B
Size: 445,287 conversations with 1,327,548 total messages
Format: Conversational data with original structure preserved
Dataset Statistics
Total… See the full description on the dataset page: https://huggingface.co/datasets/pdelobelle/nemotron-dutch-mt.dbnl.org-dutch-public-domain
Dataset Card for "dbnl.org-dutch-public-domain"
Dataset Summary
This dataset comprises a collection of texts from the Dutch Literature in the public domain, specifically from the DBNL (Digitale Bibliotheek voor de Nederlandse Letteren) public domain collection. The collection includes books, poems, songs, and other documentation, letters, etc., that are at least 140 years old and thus free of copyright restrictions. Each entry in the dataset corresponds to one section of… See the full description on the dataset page: https://huggingface.co/datasets/jvdgoltz/dbnl.org-dutch-public-domain.aya_dutch_dpo
Dataset Card for aya_dutch_dpo
This dataset has been created with distilabel.
This dataset was created as part of the Data is Better Together project, in particular as part of an ongoing effort to help foster the creation of DPO/ORPO datasets for more languages.
The dataset was constructed using the following steps:
starting with the aya_dataset and filtering for Dutch examples
using the Meta-Llama-3-70B-Instruct model to generate new examples for each promptUsing… See the full description on the dataset page: https://huggingface.co/datasets/data-is-better-together/aya_dutch_dpo.dutch_chat_datasets
Dataset Card for "dutch_chat_datasets"
This dataset is a merge of the following datasets. See their pages for licensing, usage, creation, and citation information.
https://huggingface.co/datasets/BramVanroy/dolly-15k-dutch
https://huggingface.co/datasets/BramVanroy/alpaca-cleaned-dutch-baize
https://huggingface.co/datasets/BramVanroy/stackoverflow-chat-dutch
https://huggingface.co/datasets/BramVanroy/quora-chat-dutch
They are reformatted for easier, consistent processing in… See the full description on the dataset page: https://huggingface.co/datasets/BramVanroy/dutch_chat_datasets.dolly-15k-dutch
Dataset Card for Dolly 15k Dutch
Dataset Summary
This dataset contains 14,934 instructions, contexts and responses, in several natural language categories such as classification, closed QA, generation, etc. The English original dataset was created by @databricks, who crowd-sourced the data creation via its employees. The current dataset is a translation of that dataset through ChatGPT (gpt-3.5-turbo).
☕ Want to help me out? Translating the data with the OpenAI API, and… See the full description on the dataset page: https://huggingface.co/datasets/BramVanroy/dolly-15k-dutch.DutchGovBench
DutchGovBench v0.1
Evaluation benchmark for Dutch government AI systems. 100 questions across 9 categories, testing knowledge of Dutch law and public administration.
What is this?
DutchGovBench tests whether AI models can accurately answer questions about Dutch government topics: social support law (Wmo 2015), youth law (Jeugdwet), participation law (Participatiewet), administrative law (Awb), municipal policy, objection procedures, privacy/GDPR, administrative oversight… See the full description on the dataset page: https://huggingface.co/datasets/CiviQs/DutchGovBench.aya_dataset_dutch_exampleaya_dutch_dpo_binarized
Dataset Card for aya_dutch_dpo
This dataset has been created with distilabel.
This dataset was created as part of the Data is Better Together project, in particular as part of an ongoing effort to help foster the creation of DPO/ORPO datasets for more languages.
The dataset was constructed using the following steps:
starting with the aya_dataset and filtering for Dutch examples
using the Meta-Llama-3-70B-Instruct model to generate new examples for each promptUsing… See the full description on the dataset page: https://huggingface.co/datasets/CultriX/aya_dutch_dpo_binarized.apollo_english_guidelines_translated_to_dutch_with_nllb200
Data description
Translation of the English medical guidelines that are part of the Apollo corpus, using the NLLB200-600M NTM.
Acknowledgement
The work received funding from the European Union's Horizon Europe research
and innovation programme under Grant Agreement No. 101057849 (DataTools4Heart project).
For more information on the background, see Datatools4Heart Huggingface/Website/Git
dala-dutch-dynaword
DaLA Dutch — DynaWord
Dutch grammatical acceptability and error correction with synthetic spelling and
grammar errors. Provisional, checker-screened training data; not a human-validated
gold benchmark. No simplification, paraphrasing or style-transfer task.
Configurations
478,916 original/corrupted pairs, 957,832 chat rows
per configuration. Every pair contributes a clean control and a corrupted input.
The two configurations share sentences and document splits and… See the full description on the dataset page: https://huggingface.co/datasets/schneiderkamplab/dala-dutch-dynaword.ultra_feedback_dutch_cleaned_multi
Ultra Feedback Dutch Cleaned
This dataset should not be used unless you are interest in all model generations. Instead, refer to the rated and further filtered version.
This is a cleaned version of BramVanroy/ultra_feedback_dutch, based on the cleaning done by Argilla on the original Ultra Feedback dataset.
It contains multiple LM responses from:
GEITje-7B-ultra
TowerInstruct-13B-v0.1
TowerInstruct-7B-v0.2
GEITje-7B-chat
gpt-4-turbo
llama-2-13b-chat-dutch
polish-dutch-short-utterancesThis dataset is provided by AIxBlock, an unified platform for AI development and AI workflows automation.
This dataset contains over 1 million sentences in Polish and Dutch, making it a valuable resource for a wide range of language technology applications. All data has undergone quality assurance (QA) checks to ensure clarity, correctness, and natural phrasing.
The dataset is well-suited for:
Speech data generation (e.g., recording short audio clips lasting 8–30 seconds per sentence)
Natural… See the full description on the dataset page: https://huggingface.co/datasets/AIxBlock/polish-dutch-short-utterances.