datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
reflection-50m
SPP Reflection 50M
The 51.4M-document reflection set from Synthetic Persona Pretraining (SPP):
Alignment from Token Zero — the production half-corpus run, and the dataset the
released models were actually trained on.
🔬 Small sample (same format): dlab-spp/reflection-sample-2k
📉 Earlier 10M run: dlab-spp/reflection-10m
🧾 Safety scores for the full 1T corpus: dlab-spp/safety-classifications
Each row pairs a source document with two generated constitution reflections — a… See the full description on the dataset page: https://huggingface.co/datasets/dlab-spp/reflection-50m.DSIR-filtered-pile-50M
Dataset Card for DSIR-filtered-pile-50M
Dataset Summary
This dataset is a subset of The Pile, selected via the DSIR data selection method. The target distribution for DSIR is the Wikipedia and BookCorpus2 subsets of The Pile.
Languages
English (EN)
Dataset Structure
A train set is provided (51.2M examples) in jsonl format.
Data Instances
{"contents": "Hundreds of soul music enthusiasts from the United Kingdom plan to make their way to… See the full description on the dataset page: https://huggingface.co/datasets/stanford-crfm/DSIR-filtered-pile-50M.nemo-stage1-50M-samples
NeMo Stage1 Pretraining Dataset - 50M Samples
This dataset contains 50 million text samples for NeMo model pretraining (Stage 1). The dataset is organized in chunks for efficient loading and processing.
Dataset Details
Total Samples: ~50,000,000
Format: JSONL (JSON Lines)
Structure: Each sample contains {"id": number, "text": "content"}
Chunks: 47 files (chunk_000.jsonl to chunk_046.jsonl)
Samples per chunk: ~1,000,000
Language: English
Task: Text generation pretraining… See the full description on the dataset page: https://huggingface.co/datasets/ssuresh/nemo-stage1-50M-samples.inducto-50M
Task Inducto: In-Context Rule Induction (Few-Shot Function Learning)
Overview
Task Inducto tests a language model's ability to infer a novel transformation rule from
a few examples and apply it to a new input. This is the atomic skill behind few-shot learning
and in-context learning more broadly.
Why This Task Matters
This task targets the "Induction Head" hypothesis:
Models must recognize patterns like [A][B]...[A] → predict [B]
Rules are randomly sampled… See the full description on the dataset page: https://huggingface.co/datasets/ericflo/inducto-50M.logic-50M
Task Logic: Boolean Circuit Evaluation (The "XOR" Problem)
Overview
Task Logic tests a language model's ability to evaluate boolean circuits with non-linear
gates, especially XOR. This targets the FFN/MLP layers rather than attention, testing
whether the model can perform actual computation rather than just retrieval.
Why This Task Matters
XOR/parity is the classic "hard problem" for neural networks (Minsky & Papert):
Cannot be computed by linear… See the full description on the dataset page: https://huggingface.co/datasets/ericflo/logic-50M.wildchat50m-rewild-sft-385700
wildchat50m-rewild-sft-385700
A supervised fine-tuning (SFT) dataset formed by the union of three sources, each
reformatted to a single canonical conversational schema (WildChat's format is the
ground-truth). Single train split, 385,700 rows.
This is a capped variant of
nyu-dice-lab/wildchat50m-rewild-sft-1118773:
identical union and format handling, except the WildChat source is randomly
subsampled to 250,000 rows (the other two sources are kept in full).
⚠️… See the full description on the dataset page: https://huggingface.co/datasets/nyu-dice-lab/wildchat50m-rewild-sft-385700.filtro-50M
Task Filtro: Interleaved Source Separation (The "Cocktail Party" Problem)
Overview
Task Filtro tests a language model's ability to selectively attend to one information
stream while actively suppressing a competing, interleaved noise stream. 50% of the context
is active noise that must be filtered out.
Why This Task Matters
This task specifically exposes architectural differences in noise handling:
Transformers EXCEL: Q·K attention can assign near-zero… See the full description on the dataset page: https://huggingface.co/datasets/ericflo/filtro-50M.en-bg-os-full-50m
Bulgarian-English OpenSubtitles Full Dataset (50M, ChessInstruct Format)
🎯 Overview
This dataset contains 48,749,944 English to Bulgarian subtitle translation pairs in ChessInstruct format for fine-tuning Gemma3-270m using the Unsloth framework. This represents the complete OpenSubtitles parallel corpus for the BG-EN language pair, making it one of the largest translation datasets available.
✨ Key Features
📊 Massive Scale: 48.7 million translation pairs from… See the full description on the dataset page: https://huggingface.co/datasets/zantag/en-bg-os-full-50m.dyna-50M
Task Dyna: Dynamic State Tracking (Non-Commutative Permutation Composition)
Overview
Task Dyna tests a language model's ability to maintain and update a mutable mental state
through a sequence of non-commutative operations. Unlike retrieval tasks where answers exist
in the context, Dyna's answers are latent variables that must be computed through
sequential operation composition.
Why This Task Matters
This task specifically targets the "write head"… See the full description on the dataset page: https://huggingface.co/datasets/ericflo/dyna-50M.wildchat50m-rewild-sft-1118773
wildchat50m-rewild-sft-1118773
A supervised fine-tuning (SFT) dataset formed by the union of three sources, each
reformatted to a single canonical conversational schema (WildChat's format is the
ground-truth). Single train split, 1,118,773 rows.
Canonical schema
Column
Type
Description
conversation_hash
string
Per-row identifier
conversation
list[{role: string, content: string}]
The chat turns
model
string
Provenance / generating-model label… See the full description on the dataset page: https://huggingface.co/datasets/nyu-dice-lab/wildchat50m-rewild-sft-1118773.