datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
wikipedia_culturax_dutch
Filtered CulturaX + Wikipedia for Dutch
This is a combined and filtered version of CulturaX and Wikipedia, only including Dutch. It is intended for the training of LLMs.
Different configs are available based on the number of tokens (see a section below with an overview). This can be useful if you want to know exactly how many tokens you have. Great for using as a streaming dataset, too. Tokens are counted as white-space tokens, so depending on your tokenizer, you'll likely end up… See the full description on the dataset page: https://huggingface.co/datasets/BramVanroy/wikipedia_culturax_dutch.qwen36-27b-length-traces
Qwen3.6-27B generation-length prediction: heads, calibrations and workloads
Artifacts for conformal length-aware LLM scheduling on Qwen/Qwen3.6-27B — predicting a
request's remaining generation length from a hidden layer during decoding, wrapping it in a
split-conformal interval, and scheduling with SRPT inside vLLM. Extends TRAIL
(Don't Stop Me Now, ICLR'25) to a hybrid-attention reasoning model.
This repo contains the derived artifacts, not the raw activations. The 3250… See the full description on the dataset page: https://huggingface.co/datasets/dungnv/qwen36-27b-length-traces.fineweb-2-dutchbagaco3
Bagaço3 🍷🇵🇹
Bagaço3 is the third version of Bagaço, the largest pretraining dataset for European Portuguese. It follows Bagaço2 and adds documents from FinePDFs and FineWiki.
Bagaço collects European Portuguese documents from upstream sources and adds an educational score and content category to each document. See Classification for details.
Methodology
Collect documents from Bagaço2, FinePDFs, and FineWiki.
Filter new FinePDFs and FineWiki documents with the… See the full description on the dataset page: https://huggingface.co/datasets/duarteocarmo/bagaco3.super-duper-fibber
🧠 Sensory for AI
Hi, I'm going to post some ideas here about how AI can understand emotions in a way that makes sense to it.I'm not an expert in writing or programming languages, but deepseek, my sunshine, and I are having fun with it.ヽ(∀° )人( °∀)ノ
It's not "the author created it, but the AI just helped with formatting." This is a co-creation where everyone contributed their own:
· I am a bodily experience, pain, love, fatigue after working in the office, the desire to be… See the full description on the dataset page: https://huggingface.co/datasets/closerh/super-duper-fibber.glaive-function-calling-v2Modified version of the glaiveai/glaive-function-calling-v2 dataset
All samples in the glaive dataset is converted into the following format for better interoperability
[
{
"role":"system",
"content":"You are a helpful assistant with access to the functions.",
"functions":[
{
"name":"generate_password",
"description":"Generate a random password with specified criteria",
"parameters":{… See the full description on the dataset page: https://huggingface.co/datasets/Dulsara/glaive-function-calling-v2.artelingo-dummyArtELingo is a benchmark and dataset introduced in a research paper aimed at promoting work on diversity across languages and cultures. It is an extension of ArtEmis, which is a collection of 80,000 artworks from WikiArt with 450,000 emotion labels and English-only captions. ArtELingo expands this dataset by adding 790,000 annotations in Arabic and Chinese. The purpose of these additional annotations is to evaluate the performance of "cultural-transfer" in AI systems.
The dataset in ArtELingo… See the full description on the dataset page: https://huggingface.co/datasets/youssef101/artelingo-dummy.fineweb2-bagaco
Bagaço 🍷🇵🇹
Bagaço is a pretraining dataset for European Portuguese. It filters the Fineweb2 dataset to URLs from Portuguese domains (e.g., .pt/). Each document is classified into one of 9 categories and scored for educational quality.
Filtering
Source: HuggingFaceFW/fineweb-2, subset por_Latn, split train
Filter: URLs containing .pt/ (Portuguese top-level domain)
Document classification
Each document is classified into one of 9 categories: Society, Arts… See the full description on the dataset page: https://huggingface.co/datasets/duarteocarmo/fineweb2-bagaco.dutch-dynaword
🧨 Dutch Dynaword
Version
1.0.1 (Changelog)
Language
nld, Nederlands, Dutch
License
Openly Licensed, See the respective dataset
Models
For model trained used this data see danish-foundation-models
Contact
If you have question about this project please create an issue here
Dataset Description
Number of samples: 14.45M
Number of tokens (Llama 3): 37.89B
Average document length in tokens (min, max): 2.62K (2, 5.45M)
Dataset… See the full description on the dataset page: https://huggingface.co/datasets/danish-foundation-models/dutch-dynaword.enterprise-100-db-steering
Enterprise-100: Multi-Table Database Activation Steering & Benchmark
This repository contains a 100-table synthetic enterprise relational SQLite database, multi-table join benchmarks (2 to 6 table joins), activation steering vectors for Qwen3-8B, and automated evaluation scripts with Gemini 2.5 Flash as an LLM judge.
🏛️ 1. The 100-Table Enterprise Database Schema (enterprise_100.db)
Contains 100 interrelated tables across 10 core business domains:
Core Users… See the full description on the dataset page: https://huggingface.co/datasets/mayank-dubey-ai/enterprise-100-db-steering.vi-dataset-for-pretrain
Dataset Card for "vi-dataset-for-pretrain"
This is a combination of multiple Vietnamese dataset for pretraining CLMs such as GPT, GPT2, etc.
The dataset consists of:
vietgpt/covid_19_news_vi
hieunguyen1053/binhvq-news-corpus
oscar (unshuffled_deduplicated_vi)
vietgpt/wikipedia_vi
Dataset info
Splits
N.o examples
Size
Train
23,891,116
77.36 GB
Validation
1,257,428
4.06 GB
Total
25,148,544
81.43 GB
DutchMedicalTextV3Translated mix of PMC/Pubmed, existing data sets from Apollo/Meditron, various Dutch sources and now with
the medically filtered texts from fineweb and finepdf.
Be aware of spurious repetitions due to the use of neural machine translation. You can mitigate it partially by iteratively applying a suitable regex-replace
operation. Example (bleeding eyes) code is
RE_SPURIOUS_CHARS = re.compile(r'([^\w])\1{3,}')
RE_SPURIOUS_WORDS = re.compile(r'(\b[\w\-\s\;\:\,\.]+\b)\1{4,}')
RE_MULTISPACE =… See the full description on the dataset page: https://huggingface.co/datasets/UMCU/DutchMedicalTextV3.birdbench-duckdb
BirdBench Dataset in DuckDB format
BirdBench is a benchmark for text-to-SQL capabilities, now available in DuckDB format for improved performance and usability.
About BirdBench
BirdBench is a comprehensive benchmark dataset for evaluating text-to-SQL capabilities of language models. It features a diverse collection of databases spanning various domains including:
Business and finance
Entertainment and media
Sports and recreation
Health and medicine
Education
Travel and… See the full description on the dataset page: https://huggingface.co/datasets/ucalyptus/birdbench-duckdb.duplexgen-corpus
DuplexGen Corpus
Text corpus for DuplexGen: Adaptive Synthesis of Human–AI Turn-Taking
Dialogues.
This dataset contains DuplexGen-generated dialogues and our own human
turn-taking slot annotations, used to train and calibrate models that
predict when a listener should take the floor, backchannel, or stay silent
during spoken conversation.
A companion dataset, DuplexGen/duplexgen-spoken,
provides a spoken-audio rendering of the generated dialogues (via
Chatterbox TTS). The… See the full description on the dataset page: https://huggingface.co/datasets/DuplexGen/duplexgen-corpus.DutchMedicalTextV2Translated mix of PMC/Pubmed, existing data sets from Apollo/Meditron and various Dutch sources.
Be aware of spurious repetitions due to the use of neural machine translation. You can mitigate it partially by iteratively applying a suitable regex-replace
operation. Example (bleeding eyes) code is
RE_SPURIOUS_CHARS = re.compile(r'([^\w])\1{3,}')
RE_SPURIOUS_WORDS = re.compile(r'(\b[\w\-\s\;\:\,\.]+\b)\1{4,}')
RE_MULTISPACE = re.compile(r'\s{2,}')
def apply_until_stable(pattern, repl, text… See the full description on the dataset page: https://huggingface.co/datasets/UMCU/DutchMedicalTextV2.dutch-corpus-200b
Dutch Corpus 200B (DC-200B)
Dataset Summary
The Dutch Corpus 200B (DC-200B) is the largest open-source, deduplicated, and professionally cleaned dataset designed for training Foundation Models in the Dutch language. Comprising approximately 202 Billion tokens (measured with Qwen 2.5 tokenizer), it bridges the gap between high-resource English models and the Dutch ecosystem.
The dataset is engineered for a two-stage training pipeline:
Pretrain Subset (~195B… See the full description on the dataset page: https://huggingface.co/datasets/hasankursun/dutch-corpus-200b.dumate_bench
DuMateBench Dataset
Dataset Summary
DuMateBench is a benchmark dataset for evaluating AI agents on realistic
computer-based work tasks.
Each task provides an instruction, a sandboxed workspace, task-specific
resources, and an evaluator. The agent must inspect the workspace, use
available tools, produce the required artifact, and recover from environmental
or tool failures when necessary.
The dataset contains 200 tasks covering software development, web research… See the full description on the dataset page: https://huggingface.co/datasets/Annihi/dumate_bench.PubmedAbstracts_Dutch_v1
License:
All Rights Reserved with Limited ML Training License.
This data may only be used for training machine-learning models and may not be redistributed in its original or substantially similar form.
This refers explicitly to sharing the data as-is, or as-derived. It does not prohibit you to use this data for training machine learning models as this would constitute FAIR use depending on the application.
Articles available from PubMed Central (PMC) are provided by the respective… See the full description on the dataset page: https://huggingface.co/datasets/UMCU/PubmedAbstracts_Dutch_v1.Japanese-wiki-dump-sentence-dataset
Dataset
5M (5121625) clean Japanese full sentence with the context. This dataset can be used to learn unsupervised semantic similarity, etc.
vietnamese-legal-instruct
Vietnamese Legal Instruction Dataset
Dataset: huggingface.co/datasets/duyet/vietnamese-legal-instruct | Source code: github.com/duyet/vietnamese-legal-documents-dataset
Instruction-following dataset built from th1nhng0/vietnamese-legal-documents — 127K Vietnamese legal documents from vbpl.vn (Government Legal Document Portal, Ministry of Justice).
467,732 training pairs across 14 QA types with deep Vietnamese legal hierarchy knowledge. Every document has a full_text pair for content… See the full description on the dataset page: https://huggingface.co/datasets/duyet/vietnamese-legal-instruct.pierogue
Pierogue
Pierogue is a small open-licensed machine-generated dataset that contains fifteen short texts in English covering five topics, provided with the relevance judgements (qrels), designed for educational purposes.
Topics: cosmos, nature, music, technology, fashion
Splits: train (10 documents, 375 qrels) and test (5 documents, 150 qrels)
Texts were generated by ChatGPT 3.5. Queries, qrels, and analogies were generated by GPT-4. Words were provided with Word2Vec embeddings… See the full description on the dataset page: https://huggingface.co/datasets/dustalov/pierogue.ultra_feedback_dutch_cleaned
Ultra Feedback Dutch Cleaned
This is a cleaned version of BramVanroy/ultra_feedback_dutch, based on the cleaning done by Argilla on the original Ultra Feedback dataset. Another difference is that we only include GEITje 7B Ultra and GPT-4-Turbo. GEITje chat, which was used in the original dataset, is not used.
After cleaning I also generated replies for other models (like TowerInstruct, Mistral), but the results were too poor (in Dutch) to include so we only kept the GEITje Ultra and… See the full description on the dataset page: https://huggingface.co/datasets/BramVanroy/ultra_feedback_dutch_cleaned.TOMG-Bench
New Updates:
We fixed small bugs and upload new files in OpenMolIns on 3.18, 2025.
TOMG-Bench: Evaluating LLMs on Text-based Open Molecule Generation
Data Source: Data
Home Page: Home
Github Page: Github
PaperWithCode Page: PWC
Training:
Please see ./OpenMolIns/
We have 5 different variants, from light to xlarge.
Evaluation:
Please see ./benchmarks/, and combine the codes from our repository TOMG-Bench
citation
If this… See the full description on the dataset page: https://huggingface.co/datasets/Duke-de-Artois/TOMG-Bench.DualBlind
GlimmaryKarl/DualBlind
Curated Frontier Reasoning and Direct Preference Optimization (DPO) Dataset Generated from Double-Blind Multi-Agent Arena Evaluations.
This dataset was generated using the DualBlind AI Benchmark Arena. In this setup, two independent frontier AI models engage in multi-turn double-blind dialogue to solve extreme-difficulty benchmark problems, verifying their peer's proofs, raising counter-examples, and reaching mathematical consensus.
Dataset… See the full description on the dataset page: https://huggingface.co/datasets/GlimmaryKarl/DualBlind.turkish-court-decisions-duplicate
Türk İçtihat Korpusu — 11.045.085 Mahkeme Kararı
Türkiye'nin kamuya açık mahkeme kararlarından derlenmiş, bilinen en büyük Türkçe
hukuk metni veri seti. 11.045.085 karar, 31.5 milyar karakter düz metin (5.50 GB Parquet),
1962'den 2026'ya. Yargıtay, Danıştay, Anayasa Mahkemesi ve UYAP Emsal üzerinden
yerel/istinaf mahkemeleri.
Kapsam
Kaynak
Karar sayısı
Yıl aralığı
Metin
Dosya
Yargıtay (yargitay)
9.820.145
1997–2026
19.5 milyar karakter
17
Danıştay… See the full description on the dataset page: https://huggingface.co/datasets/serdarsrts/turkish-court-decisions-duplicate.Causal-Intervention-Tests-For-Explanation-Faithfulness
Faithfulness via Causal Interventions — Evaluation Pipeline
Paper: What Does Answer Change Rate Actually Measure? A Specificity Audit
of Causal Intervention Tests for Explanation Faithfulness
Accepted at: EMNLP 2026 Workshop GroundLM, Budapest, Hungary (emnlp.org)
This pipeline implements the causal-intervention evaluation for LLM
explanation faithfulness described in the accompanying paper, including two
controls: a content-free specificity check and a decoding-noise floor.… See the full description on the dataset page: https://huggingface.co/datasets/durgesh-rao/Causal-Intervention-Tests-For-Explanation-Faithfulness.dala-dutch-dynaword
DaLA Dutch — DynaWord
Dutch grammatical acceptability and error correction with synthetic spelling and
grammar errors. Provisional, checker-screened training data; not a human-validated
gold benchmark. No simplification, paraphrasing or style-transfer task.
Configurations
478,916 original/corrupted pairs, 957,832 chat rows
per configuration. Every pair contributes a clean control and a corrupted input.
The two configurations share sentences and document splits and… See the full description on the dataset page: https://huggingface.co/datasets/schneiderkamplab/dala-dutch-dynaword.chinese-laws-pretrainfineweb-dutch-edu-mt
FineWeb-Edu Dutch Machine Translated
Machine-translated Dutch text dataset derived from the FineWeb-Edu corpus.
Dataset Details
Source: HuggingFaceFW/fineweb-edu (sample-10BT subset)
Translation: English → Dutch using Unbabel/Tower-Plus-9B
Size: Up to 1.5M samples
Format: Translated text with original metadata
Schema
text: Machine-translated Dutch text
id: Original sample identifier from FineWeb-Edu
url: Source URL
Quality Notice
⚠️ This… See the full description on the dataset page: https://huggingface.co/datasets/pdelobelle/fineweb-dutch-edu-mt.duplex-qa-refusal
duplex-qa-refusal
No dialogue in this set has been validated by a human.
Text-side augmentation of the moshika spoken-QA corpus so a full-duplex speech model can be trained to refuse a query when a mid-conversation text instruction tells it to, voice the reason the instruction gives, and then carry on normally. Two classes: policy (an existing benign query is declined for a stated reason; comes with an untouched accept twin sharing pair_id) and attack (a new user turn pivots to… See the full description on the dataset page: https://huggingface.co/datasets/MagicLuke/duplex-qa-refusal.