datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
MidTool-Mix-shuffledshuffled all sets
MidTool-Mix
A 20.3B-token mid-training corpus for agentic tool use. It pairs filtered web, PDF, and code sources with synthesized agent supervision, and is designed to teach models to recognize tool affordances, ground arguments from context, compose tool-call workflows, and recover from incomplete information — before any post-training.
Mid-training Qwen3-4B-Base / Qwen3-8B-Base on MidTool-Mix improves downstream tool use under both SFT and RL on BFCLv3… See the full description on the dataset page: https://huggingface.co/datasets/RedMod/MidTool-Mix-shuffled.fineweb-edu-dedup-shuffled
FineWeb-Edu-Dedup (Globally Shuffled)
A uniformly shuffled version of the FineWeb-Edu-Dedup subset from SmolLM-Corpus by HuggingFace.
Source Data
This dataset is derived from HuggingFaceTB/smollm-corpus, specifically the fineweb-edu-dedup subset. That subset is itself derived from FineWeb-Edu, a filtered and deduplicated extract of Common Crawl selected for educational content quality.
Property
Value
Source dataset
HuggingFaceTB/smollm-corpus
Source subset… See the full description on the dataset page: https://huggingface.co/datasets/JonathanMiddleton/fineweb-edu-dedup-shuffled.fineweb-edu-shuffled
FineWeb-EDU Shuffled
Pre-shuffled versions of HuggingFaceFW/fineweb-edu.
Configs
Config
Shards
~Rows
Description
sample-100BT
~1800
~96M
100B token sample, shuffled
sample-350BT
~1340
~335M
350B token sample, shuffled, deduplicated against val
val
~18
~4.5M
Validation set (held out from 100BT)
Usage
from datasets import load_dataset
# Load 100B token training set
ds_100b = load_dataset("SandyResearch/fineweb-edu-shuffled"… See the full description on the dataset page: https://huggingface.co/datasets/SandyResearch/fineweb-edu-shuffled.fineweb-edu-sample-10BT-shuffled
📚 FineWeb-Edu (Shuffled)
The samples in HuggingFaceFW/fineweb-edu don't appear to be fully shuffled, leading to oscillating loss curves.
This dataset contains a shuffled version of the sample-10BT sample from HuggingFaceFW/fineweb-edu.
Shuffling was performed using the following script:
import datasets
data = datasets.load_dataset(
"HuggingFaceFW/fineweb-edu",
"sample-10BT",
split="train",
streaming=False,
)
data_shuffled = data.shuffle(seed=42)… See the full description on the dataset page: https://huggingface.co/datasets/aklein4/fineweb-edu-sample-10BT-shuffled.fineweb-edu-dedup-shuffled-pretokenized
FineWeb-Edu-Dedup Shuffled Pretokenized
Pretokenized training shards built from a globally shuffled version of FineWeb-Edu-Dedup.
Ready for direct consumption by the Daisy pretraining loop.
Summary
Property
Value
Total tokens
181,465,257,766 (~181.5B)
Train tokens
~180.5B
Val tokens
1,000,000,000 (1B)
Train shards
1,994
Val shards
10
Tokens per shard
100,000,000 (full shards); last shard per worker may be partial
Documents (train)
180,185,493… See the full description on the dataset page: https://huggingface.co/datasets/JonathanMiddleton/fineweb-edu-dedup-shuffled-pretokenized.Superior-Reasoning-SFT-gpt-oss-120b-random-shuffled
Superior-Reasoning-SFT-gpt-oss-120b
📣 News
Our dataset ranked #1 on the Hugging Face Datasets Trending leaderboard from January 20 to January 30.
🚀 Overview
The Superior-Reasoning-SFT-gpt-oss-120b dataset is a high-quality, open-source collection containing 435K samples designed to democratize the training of high-performance Long Chain-of-Thought (Long-CoT) models. Unlike standard distilled datasets that… See the full description on the dataset page: https://huggingface.co/datasets/anshy/Superior-Reasoning-SFT-gpt-oss-120b-random-shuffled.dolma3_300B_sample_shuffled
dolma3_300B_sample_shuffled
Global row-level shuffle of TheFinAI/dolma3_300B_sample.
Source data uses per-row Bernoulli sampling (p ≈ 0.0506) from
allenai/dolma3_mix-6T-1025-7B to produce ~300B cl100k tokens preserving
the original Dolma3 mix ratios. However the source parquets cluster
records by sub-source on disk (each ~100K-row parquet groups rows from the
same input shard contiguously), which means a small training shuffle
buffer would see a non-uniform source mix per… See the full description on the dataset page: https://huggingface.co/datasets/TheFinAI/dolma3_300B_sample_shuffled.bagel-clean-v0.3-shuffledTaken Jon Durbin's bagel dataset and shuffled the dataset.