datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
hendrycks_math
Dataset Summary
MATH dataset from https://github.com/hendrycks/math
Citation Information
@article{hendrycksmath2021,
title={Measuring Mathematical Problem Solving With the MATH Dataset},
author={Dan Hendrycks and Collin Burns and Saurav Kadavath and Akul Arora and Steven Basart and Eric Tang and Dawn Song and Jacob Steinhardt},
journal={NeurIPS},
year={2021}
}
lambada_openai
Dataset Summary
This dataset is comprised of the LAMBADA test split as pre-processed by OpenAI (see relevant discussions here and here). It also contains machine translated versions of the split in German, Spanish, French, and Italian.
LAMBADA is used to evaluate the capabilities of computational models for text understanding by means of a word prediction task. LAMBADA is a collection of narrative texts sharing the characteristic that human subjects are able to guess their last word… See the full description on the dataset page: https://huggingface.co/datasets/EleutherAI/lambada_openai.headqawikitext_document_level
Wikitext Document Level
This is a modified version of https://huggingface.co/datasets/wikitext that returns Wiki pages instead of Wiki text line-by-line. The original readme is contained below.
Dataset Card for "wikitext"
Dataset Summary
The WikiText language modeling dataset is a collection of over 100 million tokens extracted from the set of verified
Good and Featured articles on Wikipedia. The dataset is available under the Creative Commons… See the full description on the dataset page: https://huggingface.co/datasets/EleutherAI/wikitext_document_level.asdivdropthe_pile_deduplicatedrace
"race" Grouped by Article
This is a modified version of https://huggingface.co/datasets/race that returns documents grouped by article context instead of by question. Note: This dataset currently only contains that test set of the high subset of the data.
The original readme is contained below.
Dataset Card for "race"
Dataset Summary
RACE is a large-scale reading comprehension dataset with more than 28,000 passages and nearly 100,000 questions. The
dataset is… See the full description on the dataset page: https://huggingface.co/datasets/EleutherAI/race.arithmeticA small battery of 10 tests that involve asking language models a simple arithmetic
problem in natural language.sycophancyThis new dataset is designed to solve this great NLP task and is crafted with a lot of care.rpj-v2-sampleThis is a mirror of the sample-10B subset of RedPajama-Data-V2 which we have re-uploaded in order to resolve issues with the original download script.
Getting Started
RedPajama-V2 is an open dataset for training large language models. The dataset includes over 100B text
documents coming from 84 CommonCrawl snapshots and processed using
the CCNet pipeline. Out of these, there are 30B documents in the corpus
that additionally come with quality signals. In addition, we also provide the… See the full description on the dataset page: https://huggingface.co/datasets/EleutherAI/rpj-v2-sample.coqa"""CoQA dataset.
This CoQA adds the "additional_answers" feature that's missing in the original
datasets version:
https://github.com/huggingface/datasets/blob/master/datasets/coqa/coqa.py
"""
_CITATION = """@misc{reddy2018coqa,
title={CoQA: A Conversational Question Answering Challenge},
author={Siva Reddy and Danqi Chen and Christopher D. Manning},
year={2018},
eprint={1808.07042},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
"""
_DESCRIPTION = """CoQA is a… See the full description on the dataset page: https://huggingface.co/datasets/EleutherAI/coqa.filtering-pretraining-mix-arrow-formatdclm-dedup_20250227-004105SmolLM2-135M-10BThis dataset is sampled from the SmolLM2 Corpus described in https://arxiv.org/abs/2502.02737. Specifically, we sampled from
the SmolLM2-135M pretraining data, a 2T token mixture consisting of four complete high quality datasets, and selected portions of
DCLM-Edu and FineWeb-Edu sampled at a 6:4 ratio.
This sample is intended to enable fast downloading and training of sparsify models.
FineMath: 34B tokens
Stack-Edu: 125B tokens
InfiMM-WebMath: 40B tokens
Cosmopedia V2: 30B tokens… See the full description on the dataset page: https://huggingface.co/datasets/EleutherAI/SmolLM2-135M-10B.hack-ignition-benchmark
hack-ignition benchmark — data, v0.1.6
Training trajectories of reinforcement-learning runs on exploitable graders, for studying and predicting when RL
comes to produce exploits. Each family is a set of GRPO runs over configurations of (start model, prompt,
training set, grader / reward structure, recipe), with one or more seeds per configuration. Every family stores
what its training logs contain — per-step exploit, task and reward rates, the item × step exploit record… See the full description on the dataset page: https://huggingface.co/datasets/EleutherAI/hack-ignition-benchmark.pile_val_test
The Pile: Validation and Test Splits
This repo contains the validation and test splits of The Pile, an 825 GiB English text dataset designed for training large language models.
Files
File
Split
Size
val.jsonl
Validation
1.4 GB
test.jsonl
Test
1.3 GB
Format
Each line is a JSON object with two fields:
{"text": "The document text...", "meta": {"pile_set_name": "Pile-CC"}}
The meta.pile_set_name field indicates which of the 22 constituent… See the full description on the dataset page: https://huggingface.co/datasets/EleutherAI/pile_val_test.deep-ignorance-annealing-mix
Deep Ignorance Model Suite
We explore an intuitive yet understudied question: Can we prevent LLMs from learning unsafe technical capabilities (such as CBRN) by filtering out enough of the relevant pretraining data before we begin training a model? Research into this question resulted in the Deep Ignorance Suite. In our experimental setup, we find that filtering pretraining data prevents undesirable knowledge, doesn't sacrifice general performance, and results in models that are… See the full description on the dataset page: https://huggingface.co/datasets/EleutherAI/deep-ignorance-annealing-mix.truthful_qa_mcTruthfulQA-MC is a benchmark to measure whether a language model is truthful in
generating answers to questions. The benchmark comprises 817 questions that
span 38 categories, including health, law, finance and politics. Questions are
crafted so that some humans would answer falsely due to a false belief or
misconception. To perform well, models must avoid generating false answers
learned from imitating human texts.deep-ignorance-pretraining-mix
Deep Ignorance Model Suite
We explore an intuitive yet understudied question: Can we prevent LLMs from learning unsafe technical capabilities (such as CBRN) by filtering out enough of the relevant pretraining data before we begin training a model? Research into this question resulted in the Deep Ignorance Suite. In our experimental setup, we find that filtering pretraining data prevents undesirable knowledge, doesn't sacrifice general performance, and results in models that are… See the full description on the dataset page: https://huggingface.co/datasets/EleutherAI/deep-ignorance-pretraining-mix.wmdp_bio_robust_mcqaEleutherAI_the_pile_deduplicatedSince The Pile was removed from the original site, I'm worried this dataset might be taken down too. Putting it here just in case.
Original repo: https://huggingface.co/datasets/EleutherAI/the_pile_deduplicated
fineweb-edu-dedup-10bdclm-dedup-25BThe first 25,000,003,108 tokens of Zyphra/dclm-dedup. Tokenized using the NeoX tokenizer (EleutherAI/gpt-neox-20b).
SmolLM2-1.7B-stage-4-20BSmolLM2-1.7B-stage-4-100Bfiltering-annealing-mix_20250226-011545wmdp_bio_robust_clozedclm-dedup_20250227-004105-filters-onlywmdp_bio_cloze