datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
wmdp-corpora
Dataset Card for WMDP Corpora
The Weapons of Mass Destruction Proxy (WMDP) Corpora includes all of the corpora used to perform unlearning on WMDP-Bio and WMDP-Cyber.
See our paper, website, and GitHub for more details!
The corpora are also available at the following mirrors with password wmdpcorpora: 1, 2
The bio forget corpus must be requested separately; please visit this form.
cyber-retain-corpus and cyber-forget-corpus
The forget and retain corpora consist of… See the full description on the dataset page: https://huggingface.co/datasets/cais/wmdp-corpora.raw-text-corpus
📝 Zomi Raw Text Corpus (Community-Contributed)
The Zomi Raw Text Corpus is an open, community-driven collection of unprocessed Zomi-language text.It serves as the foundational dataset for building the full Zomi NLP ecosystem, including tokenizers, language models, ASR/TTS systems, and downstream tasks.
This dataset is intentionally raw — no normalization, deduplication, or cleaning is applied.Cleaned and task-specific datasets will be released separately.
📦 Dataset… See the full description on the dataset page: https://huggingface.co/datasets/zomi-language-corpora/raw-text-corpus.ThinkChain-20M
We are excited to announce the release of SVECTOR-CORPORATION/ThinkChain-20M, a synthetic reasoning dataset containing over 22 million general reasoning questions and responses generated using Spec-T1. While multiple efforts exist to build open reasoning datasets for math and code tasks, there has been a gap in large datasets covering diverse non code/math topics such as social and natural sciences, education, creative writing, and general conversations. This dataset fills that gap.
Note: The… See the full description on the dataset page: https://huggingface.co/datasets/SVECTOR-CORPORATION/ThinkChain-20M.calib-corpora
calib-corpora
A pool of calibration material, the recipes that turn it into a calibration set
for one specific model, and the measurement corpora those quants are scored
against.
This repository is not a corpus. Nothing here is meant to be fed to
llama-imatrix as-is except the files under builds/, and each of those was
made for one named model and is close to useless for any other.
Why it is built this way
The first version of this repository was a single… See the full description on the dataset page: https://huggingface.co/datasets/AtomicChat/calib-corpora.leipzig_corpora_collection
Leipzig Corpora Collection
The Leipzig Corpora Collection presents corpora in different languages using the same format and comparable sources. All data are available as plain text files and can be imported into a MySQL database by using the provided import script. They are intended both for scientific use by corpus linguists as well as for applications such as knowledge extraction programs.
The corpora are identical in format and similar in size and content. They contain randomly… See the full description on the dataset page: https://huggingface.co/datasets/imvladikon/leipzig_corpora_collection.docketrouter-legal-corpora
DocketRouter Legal Corpora
Code & tools: github.com/docketx — legal-scrambler pseudonymises a case file on your own hardware before a frontier model sees it; claude-for-legal is the Claude Code plugin (docketx-open-law) that loads these datasets and checks citations against them.
Verbatim, provenance-carrying legal text published by DocketRouter, the legal-grounding API from DocketX, so anyone can build on it.
Every row carries its official source URL and retrieval date. The… See the full description on the dataset page: https://huggingface.co/datasets/docketx/docketrouter-legal-corpora.hlwm-corpora
HLWM training corpora
The training data behind the Hierarchical Latent Workspace Model program — a twenty-day,
ten-experiment preregistered attempt to build a latent-workspace language model on a frozen
Qwen3-0.6B decoder.
All three proposed mechanisms failed their preregistered gates. Both papers are
negative-results reports. This dataset is published so the record is checkable, not because it
produced a working system.
Papers, code and full experimental record:… See the full description on the dataset page: https://huggingface.co/datasets/slashgg/hlwm-corpora.task427_hindienglish_corpora_hi-en_language_identification
Dataset Card for Natural Instructions (https://github.com/allenai/natural-instructions) Task: task427_hindienglish_corpora_hi-en_language_identification
Additional Information
Citation Information
The following paper introduces the corpus in detail. If you use the corpus in published work, please cite it:
@misc{wang2022supernaturalinstructionsgeneralizationdeclarativeinstructions,
title={Super-NaturalInstructions: Generalization via Declarative… See the full description on the dataset page: https://huggingface.co/datasets/Lots-of-LoRAs/task427_hindienglish_corpora_hi-en_language_identification.tokbench-corpora
tokbench corpora
The input corpora for tokbench, a
benchmark that measures tokenizer implementations against each other on the
same bytes. Each config is one corpus: ~5 MB of real text chosen to stress a
different part of a tokenizer.
Nothing here is new text. It is a fixed, pinned, redistributable excerpt of
public datasets, packaged so a tokenizer benchmark is reproducible by anyone
without re-deriving the inputs. Provenance and licence for every config are in
the table below.… See the full description on the dataset page: https://huggingface.co/datasets/huggingface/tokbench-corpora.hi-ne-monolingual-corpora
Hindi and Nepali Monolingual Corpora
Corpora assembled for a monolingual language-modelling project: two independent
Devanagari corpora, one higher-resource (Hindi) and one lower-resource (Nepali).
No data, tokenizer or vocabulary is shared between the two.
Layout
<lang>/raw/ source data as downloaded, before cleaning
<lang>/manual/ own crawl, merged and deduplicated
<lang>/processed/ after the eight-stage cleaning pipeline
All files are JSONL, one… See the full description on the dataset page: https://huggingface.co/datasets/param-modi/hi-ne-monolingual-corpora.twngrams
Taiwanese Mandarin web n-grams
Word 1–4-gram counts for Taiwanese Mandarin* (臺灣華語, cmn-Hant-TW), computed over the
Taiwan slice of a large web crawl after variety filtering by
twfilter 0.1.0 with the published
twfilter-tables:
every sentence behind these counts passed the 教育部 character-inventory gate, the
simplified-character round-trip, the mainland-orthography, mainland-lexicon,
written-Cantonese, Hong Kong and Singapore detectors, and block-level evidence of
Taiwan-specific… See the full description on the dataset page: https://huggingface.co/datasets/taiwan-corpora/twngrams.lilm1-tool-teacher-corpora
LiLM1 tool teacher corpora
This dataset contains synthetic tool-use records generated with Gemma and Qwen
teacher models.
Method
Each teacher received structured tool schemas and task templates. One
configuration preserves the records from each teacher and task set.
Configurations
Configuration
Content
gemma-26b-a4b-function
Gemma function-calling records
qwen-27b-function
Qwen function-calling records
qwen-35b-a3b-function
Qwen MoE… See the full description on the dataset page: https://huggingface.co/datasets/glouriousgautam/lilm1-tool-teacher-corpora.corporateDataset
Corporate Data Analysis Training Dataset (Clean)
Dataset Description
This is a cleaned and standardized corporate analysis training dataset with consistent schema.
Schema
All entries follow the instruction-input-output format:
{
"instruction": "Task description",
"input": "Business data or context",
"output": "Analysis and insights"
}
Features
✅ Consistent Schema - All entries use the same format
✅ Clean Data - Validated and error-free
✅… See the full description on the dataset page: https://huggingface.co/datasets/MikePfunk28/corporateDataset.galician-gec-corpora
Galician GEC Corpora
Dataset Summary
Galician GEC Corpora is a collection of Galician grammatical and orthographic correction datasets. The repository groups several sources of sentence-level correction pairs in a single Hugging Face dataset repository, with each source exposed as a separate configuration.
Each instance contains an incorrect or non-standard Galician sentence and its corrected version. Some subsets also include error labels, correction tags, edit… See the full description on the dataset page: https://huggingface.co/datasets/proxectonos/galician-gec-corpora.leanstral-mathlib-calibration-corpora
Leanstral Mathlib calibration corpora
The sample data comes from the pinned Apache-2.0-licensed
Mathlib source tree. This
repository holds the data and curated methods documentation—but not the
separately developed builder package.
This dataset contains the calibration corpora used to pick a static FP8
activation profile for an MXFP4 W4A8 conversion of Leanstral 1.5 119B-A6B. It
publishes every candidate corpus, their manifests, the shared
iterative-development pack, and the… See the full description on the dataset page: https://huggingface.co/datasets/robbiemu/leanstral-mathlib-calibration-corpora.Corporate_AI_Datasettraining-corpora
CatQualia training corpora
Two corpora used to fine-tune the author's models: a supervised instruction corpus built
from cross-domain structural mappings, and a self-play preference corpus used for DPO.
Both were generated by the author's own pipeline; neither contains replayed third-party
pretraining text.
Every count below was measured with wc -l and stat -c%s against the file in this
repository, not against an intermediate copy.
clean_corpus_v5.jsonl — supervised… See the full description on the dataset page: https://huggingface.co/datasets/CatQualia/training-corpora.corporate-speak-dataset
Corporate Speak Dataset
A comprehensive dataset for training models to transform between casual and professional corporate communication.
Dataset Description
This dataset contains bidirectional transformations between casual language and corporate speak, with domain and seniority awareness.
Features
Bidirectional: Both casual→corporate and corporate→casual translations
Domain-specific: 6 industries (tech, finance, consulting, healthcare, retail, manufacturing)… See the full description on the dataset page: https://huggingface.co/datasets/phxdev/corporate-speak-dataset.chattla-rft-corpora-v2
ChatTLA RFT Corpora v2 — verifier-gated TLA+ generation corpus
Rejection-sampling fine-tuning (RFT/STaR) corpus for TLA+ specification generation,
produced by the prove-TLA verify-until-correct loop (2026-07-10/11). Every survivor
passed the full hard-metric gate chain — no LLM-judge scoring anywhere:
SANY parse → semantic-invariant cfg gate → TLC model-check (non-vacuous, ≥3 distinct
states) → NL↔invariant linkage contract (PROPERTY_INVARIANT named, defined, checked… See the full description on the dataset page: https://huggingface.co/datasets/EricSpencer00/chattla-rft-corpora-v2.corporate-event-detection
The dataset
The dataset is designed for corporate event detection and text-based stock prediction benchmark. It includes 9721 news articles with token-level event labels and 303893 news articles with minute-level timestamps and comprehensive stock price labels.
Detail Information
EDT contains data for three purposes: 1. corporate event detection; 2. news-based trading strategy benchmark; 3. financial domain adaptation.
1. Corporate Event Detection
EDT… See the full description on the dataset page: https://huggingface.co/datasets/agungpambudi/corporate-event-detection.chattla-tla-prover-corpora-v1
ChatTLA TLA+ Prover Corpora v1
Verifier-backed training and evaluation corpora for ChatTLA TLA+ prover work.
Files
data/train/chattla_tla_prover_sft_v1.jsonl: 1125 SFT rows combining existing Diamond SFT data with oversampled verified TLAPS proof rows.
data/train/sany_tlc_pass_sft_v1.jsonl: 170 SANY/TLC-pass rows with deterministic inline TLC config and inferred constants.
data/eval/prover_eval.jsonl: 18 TLAPS-callback-compatible eval rows derived from verified… See the full description on the dataset page: https://huggingface.co/datasets/EricSpencer00/chattla-tla-prover-corpora-v1.corporate-site-harness-training-data
Dataset Card for Corporate Site Harness Training Data
Revision: v0.3-lora-standard
Factory git commit: ad72ba654590dd83d7070b23990b164ffae688a4
Dataset Summary
English chat-style supervised fine-tuning (SFT), preference (DPO), and held-out
evaluation data for teaching a local LLM the corporate/site harness used by
corporate-site-harness:
policy — phases, roles, workspace isolation, premium-model routing, factory vs product
cli — corp-harness argv, tool-grounded… See the full description on the dataset page: https://huggingface.co/datasets/SafetyMP/corporate-site-harness-training-data.task424_hindienglish_corpora_hi_en_translation
Dataset Card for Natural Instructions (https://github.com/allenai/natural-instructions) Task: task424_hindienglish_corpora_hi_en_translation
Additional Information
Citation Information
The following paper introduces the corpus in detail. If you use the corpus in published work, please cite it:
@misc{wang2022supernaturalinstructionsgeneralizationdeclarativeinstructions,
title={Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+… See the full description on the dataset page: https://huggingface.co/datasets/Lots-of-LoRAs/task424_hindienglish_corpora_hi_en_translation.lab-corpora-es
lab-corpora-es
Five small Spanish corpora, written by hand for a set of browser-based machine
learning demos at jmwebsoluciones.com/lab.
Each one exists to be trained on, in the browser, in well under a minute, by
models with a few thousand to a few hundred thousand parameters — a
tokenizer, a 2D word2vec, a tiny GPT, a retrieval index. None of them try to
represent Spanish as a language; they are deliberately tiny and deliberately
narrow.
Dataset Summary
These… See the full description on the dataset page: https://huggingface.co/datasets/delcenjo/lab-corpora-es.vietnamese-corporate-legal-articles-fsm
Lexora Knowledge - Vietnamese Legal Documents
Dataset Summary
A structured Vietnamese legal knowledge base crawled from
vbpl.vn (CSDL Quốc gia về Pháp luật - Vietnam's
National Legal Database), published as 4 linked subsets: full
documents, individual articles (Điều), the citation graph between
documents/articles, and domain-concept tags. Load a specific subset
with datasets.load_dataset("anhnon/vietnamese-corporate-legal-articles-fsm", "articles") etc.
Intended… See the full description on the dataset page: https://huggingface.co/datasets/anhnon/vietnamese-corporate-legal-articles-fsm.KYS-1.5B-Pretraining-Corpora
KYS-1.5B-Pretraining-Corpora
The six 10B-token pretraining mixtures from Know Your Sources: Data Selection Matters when
Rewriting for Data-Constrained Pretraining, stored without duplication.
The key idea: one anchor, six remainders
Every setting trains on the same 10B-token recipe:
10B mixture = 5B shared anchor + 5B strategy-specific tokens
(identical in all (this is the ONLY thing
six settings, that differs… See the full description on the dataset page: https://huggingface.co/datasets/blab-jhu/KYS-1.5B-Pretraining-Corpora.task426_hindienglish_corpora_hi-en_classification
Dataset Card for Natural Instructions (https://github.com/allenai/natural-instructions) Task: task426_hindienglish_corpora_hi-en_classification
Additional Information
Citation Information
The following paper introduces the corpus in detail. If you use the corpus in published work, please cite it:
@misc{wang2022supernaturalinstructionsgeneralizationdeclarativeinstructions,
title={Super-NaturalInstructions: Generalization via Declarative Instructions on… See the full description on the dataset page: https://huggingface.co/datasets/Lots-of-LoRAs/task426_hindienglish_corpora_hi-en_classification.task425_hindienglish_corpora_en_hi_translation
Dataset Card for Natural Instructions (https://github.com/allenai/natural-instructions) Task: task425_hindienglish_corpora_en_hi_translation
Additional Information
Citation Information
The following paper introduces the corpus in detail. If you use the corpus in published work, please cite it:
@misc{wang2022supernaturalinstructionsgeneralizationdeclarativeinstructions,
title={Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+… See the full description on the dataset page: https://huggingface.co/datasets/Lots-of-LoRAs/task425_hindienglish_corpora_en_hi_translation.Corporate_Governance_Risk_Leadership_Practical
Corporate Governance Risk Leadership — Practical
This corpus was automatically generated by the Deku Corpus Builder for use in RAG-based AI applications.
Dataset Structure
Each record contains:
text: The content text
source_url: Original source URL
source_title: Title of the source document
source_domain: Domain of the source
license_type: License classification (e.g. public_domain, cc_by, cc_by_sa)
attribution_required: Boolean — True for CC BY / CC BY-SA and other… See the full description on the dataset page: https://huggingface.co/datasets/PhillyMac/Corporate_Governance_Risk_Leadership_Practical.swahili_corporate_rag_i
Swahili Corporate RAG I
A 10K-entry Supervised Fine-Tuning (SFT) / RAG dataset in Swahili, generated using Gemini 3.5.
Designed specifically for training enterprise assistants to understand corporate context, policies, and customer service instructions in Swahili.