datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
FIM-Midtraining-400K
FIM-Midtraining-400K
📄 Paper · 💻 GitHub · 🤗 Collection
The mid-training corpus of "Function-Aware Fill-in-the-Middle as Mid-Training for Coding Agent Foundation Models": 400K function-aware FIM samples (~2.6B tokens under the Qwen2.5-Coder tokenizer) drawn from 75,568 Python files across 968 permissively-licensed GitHub repositories, fully decontaminated against SWE-Bench.
A coding agent's inner loop — act → observe → continue — is structurally isomorphic to a function call… See the full description on the dataset page: https://huggingface.co/datasets/TIGER-Lab/FIM-Midtraining-400K.openthoughts3-en-ar-midtrain
openthoughts3-en-ar-midtrain
Arabic translation of the OpenThoughts3_1.2M split of smoltalk2 (config Mid): long mathematical reasoning traces with <think> blocks, in a two-message user/assistant format. Translated with google/gemma-4-12B-it (bf16, greedy) on A100s. All 1,135,104 source rows are present, none dropped.
The pipeline segments each message into prose and verbatim blocks (code, LaTeX, tables, and inline non-translatables are masked and never sent to the model)… See the full description on the dataset page: https://huggingface.co/datasets/SultanR/openthoughts3-en-ar-midtrain.nemotron-mc-en-ar-midtrain
nemotron-mc-en-ar-midtrain
Arabic translation of the Nemotron-Pretraining-Multiple-Choice config of Nemotron-Pretraining-Specialized-v1.2 (pinned revision 807afc1). Translated with google/gemma-4-12B-it (bf16, greedy) on A100s. All 23,926,492 source rows are present, none dropped. English source and Arabic translation sit in the same row, so the dataset serves as a parallel corpus as well as an Arabic one. A sibling corpus from the same pipeline is available at… See the full description on the dataset page: https://huggingface.co/datasets/SultanR/nemotron-mc-en-ar-midtrain.nemotron-r1-en-ar-midtrain
nemotron-r1-en-ar-midtrain
Arabic translation of the Llama_Nemotron_Post_Training_Dataset_reasoning_r1 split of smoltalk2 (config Mid, pinned revision fc6cc21): reasoning traces with <think> blocks in a conversational format. Translated with RedHatAI/gemma-4-26B-A4B-it-FP8-dynamic (greedy) on H100s. FP8 was verified lossless against its bf16 parent before the run (chrF 96.4, 0 of 510 chunks materially diverged). All 3,644,790 source rows are present, none dropped. Sibling… See the full description on the dataset page: https://huggingface.co/datasets/SultanR/nemotron-r1-en-ar-midtrain.d24-midtrain-olmo3-10b-wholedoc
d24 Midtrain — OLMo-3 Dolmino (10B, whole-doc)
A 9.3B-token reproduction of OLMo 3's Dolma-3 Dolmino mid-train mix, built by
taking a fraction of each component of
allenai/dolma3_dolmino_mix-100B-1025.
No length filter and no chunking — every document is kept whole (some are very long: tens of
thousands of tokens). Each component reaches its target, so the realized mix matches OLMo-3's true
proportions (the OLMo-3 target % column ≈ the kept share). For training, the Megatron… See the full description on the dataset page: https://huggingface.co/datasets/sfanm/d24-midtrain-olmo3-10b-wholedoc.d24-midtrain-olmo3-5b
d24 Midtrain — OLMo-3 Dolmino (5B, chunked)
A 5.00B-token reproduction of OLMo 3's Dolma-3 Dolmino mid-train mix, built by
taking a fraction of each component of
allenai/dolma3_dolmino_mix-100B-1025.
Unlike the smaller d24-midtrain-olmo3
(which dropped documents over 2048 tokens — silently zeroing the long reasoning-trace and PDF
components), this build chunks long documents into 2048-token windows (decoded back to text),
so every component keeps ~100% of its tokens and reaches… See the full description on the dataset page: https://huggingface.co/datasets/sfanm/d24-midtrain-olmo3-5b.d24-midtrain-olmo3-5b-wholedoc
d24 Midtrain — OLMo-3 Dolmino (5B, whole-doc)
A 4.71B-token reproduction of OLMo 3's Dolma-3 Dolmino mid-train mix at its exact
component proportions, built by taking a fraction of each component of
allenai/dolma3_dolmino_mix-100B-1025.
Documents are kept whole — no length filter, no chunking.
Why whole-doc: for packed pretrain/midtrain the trainer (e.g. Megatron preprocess_data --append-eod
GPTDataset) already concatenates documents and slices them into context-length windows… See the full description on the dataset page: https://huggingface.co/datasets/sfanm/d24-midtrain-olmo3-5b-wholedoc.d24-midtrain-olmo3
d24 Midtrain — OLMo-3 Dolmino subsample
A length-filtered (≤2048 GPT-2 tokens) subsample of OLMo 3's Dolma-3 Dolmino mid-train mix,
used for the d24 v1base-olmo3 midtrain. Built by taking a uniform fraction of each
component of allenai/dolma3_dolmino_mix-100B-1025
(so the mix proportions are preserved), then dropping documents >2048 tokens — which naturally
shrinks the long-doc components (reasoning traces, olmOCR PDFs) that don't fit the 2048 context.
5,902,548 documents… See the full description on the dataset page: https://huggingface.co/datasets/sfanm/d24-midtrain-olmo3.cooperdata-sft-midtrain
CooperData — bucketed (SFT / mid-training / discarded)
Unified, coordination-quality-bucketed view of the CooperBench cooperative coding-agent datasets. Each row is one coop pair (two agents each implementing a feature in a shared repo), normalized to a common schema with a source column.
Splits
split
bucket
meaning
rows
sft
A
exemplary coordination workflow worth imitating
2455
midtraining
B
coordination present but thin / one-sided / synthetic
3849… See the full description on the dataset page: https://huggingface.co/datasets/CooperBench/cooperdata-sft-midtrain.cooperdata-v3-midtrain-blend
CooperData v3 — Midtraining Blend (Qwen3.5-9B cooperative SWE agents)
All-token midtraining mixture that bridges Qwen/Qwen3.5-9B (instruct) toward the cooperative
multi-agent SWE-coding SFT distribution. One document per row (text, tagged by source) — NOT
packed — so trl.SFTTrainer(packing=False) tokenizes per-doc and the Gated-DeltaNet recurrence
stays per-document. ~390M tokens.
Composition
source
tokens
share
role
web
210.0M
54%
general
math
55.0M… See the full description on the dataset page: https://huggingface.co/datasets/CooperBench/cooperdata-v3-midtrain-blend.Mid-Training_data_of_separate_domains
Breaking the Data Barrier – Building GUI Agents Through Task Generalization
This is the official dataset repository of GUIMid
1. Data Overview
AgentBoard is composed of 9 diverse tasks: 7 vision and language tasks and 4 lanuage only tasks.
The performances of different domains as mid-training data are as follows:
Domains
Observation
WebArena (PR)
WebArena (SR)
AndroidWorld (SR)
GUI Post-Training Only
Image
26.3
6.2
9.0
Public Baselines
GPT-4o-2024-11-20
Image… See the full description on the dataset page: https://huggingface.co/datasets/MidGUI/Mid-Training_data_of_separate_domains.lean-repository-midtraining-v1
Lean 4 repository midtraining corpus v1
This is a causal language-model corpus curated from pinned Lean 4 repositories. It is
intended for repository midtraining after introductory Lean language SFT and before
verified proof SFT or verifier-guided RL. The rows contain source text, not
instruction/answer conversations.
Dataset
Split
Chunks
Train
18,367
Validation
1,071
Total
19,438
The source-preserving builder estimates 16.71M tokens using four… See the full description on the dataset page: https://huggingface.co/datasets/Pradheep1647/lean-repository-midtraining-v1.cooperdata-bridge2x-midtrain-blend
CooperData bridge2x — Midtraining Blend (Qwen3.5-9B cooperative SWE agents)
All-token midtraining mixture (recipe bridge2x) that bridges Qwen/Qwen3.5-9B (instruct)
toward the cooperative multi-agent SWE-coding SFT distribution. One document per row (text,
tagged by source) — NOT packed — so trl.SFTTrainer(packing=False) tokenizes per-doc and the
Gated-DeltaNet recurrence stays per-document. ~200M tokens.
Composition
source
tokens
share
role
coop
120.1M… See the full description on the dataset page: https://huggingface.co/datasets/CooperBench/cooperdata-bridge2x-midtrain-blend.v11-cells-midtrain-corpus
v11 cells mid-training corpus
The delegating arm of a paired experiment: teach a 115M model to call an external
tool for arithmetic rather than to memorise the answers. Its partner, the maths-only
arm, teaches the same model to absorb the arithmetic into its weights instead.
Pre-tokenized against the v11 tokenizer
(10dd5110…, vocab 71,260), for
chrishayuk/v11-tinystories-115m-base.
Identity: 2115d6aeff3428e217ef2903a8030facd511dcb00183e9fc3faaf49d01038767
(chuk-datasets… See the full description on the dataset page: https://huggingface.co/datasets/chrishayuk/v11-cells-midtrain-corpus.d24-midtrain-math-pool
d24 Midtrain — Math Pool
The math/reasoning text used for the continued-pretrain ("midtrain") stage of the
d24 (0.757B, nanochat-style depth-24) replication. This is the shared content for
two midtrain recipes — v2-reasoning and v1base-mathheavy — which differ only in
their blend weights and in a ClimbMix web anchor (public
karpathy/climbmix-400b-shuffle,
not re-hosted here).
2,404,691 documents, ~1270M GPT-2 tokens of plain text.
Each row: source (str) and text (str).… See the full description on the dataset page: https://huggingface.co/datasets/sfanm/d24-midtrain-math-pool.italian-culture-midtrain-v2
Italian Culture Midtraining v2
Private, viewer-friendly continued-pretraining corpus for
mii-llm/zagreus-0.4B-ita. Repository: FinancialSupport/italian-culture-midtrain-v2.
split
rows
words
target-tokenizer tokens (EOS included)
train
84,031
23,357,466
48,463,711
validation
780
200,197
422,661
Viewer schema
text is the exact model-visible text. title, category, categories,
source, source_url, license, revision/document IDs, word/token counts,
and… See the full description on the dataset page: https://huggingface.co/datasets/FinancialSupport/italian-culture-midtrain-v2.cooperdata-sft-midtrain-v2
CooperData v2 — contamination-free, coordination-quality-bucketed
Unified view of the CooperBench cooperative coding-agent datasets (+ cooperative-game logs), one row per coop pair, for training a 9B model to be better at CooperBench.
Train/test safety: every pair whose (repo, task_id) is one of the 30 held-out CooperBench benchmark tasks is hard-excluded (X) before bucketing — zero benchmark leakage. team-trajectories and the codex team-coop/cmp-full-team* arms were 100% on… See the full description on the dataset page: https://huggingface.co/datasets/CooperBench/cooperdata-sft-midtrain-v2.cooperdata-bridge-midtrain-blend
CooperData bridge — Midtraining Blend (Qwen3.5-9B cooperative SWE agents)
All-token midtraining mixture (recipe bridge) that bridges Qwen/Qwen3.5-9B (instruct)
toward the cooperative multi-agent SWE-coding SFT distribution. One document per row (text,
tagged by source) — NOT packed — so trl.SFTTrainer(packing=False) tokenizes per-doc and the
Gated-DeltaNet recurrence stays per-document. ~100M tokens.
Composition
source
tokens
share
role
coop
60.0M
60%… See the full description on the dataset page: https://huggingface.co/datasets/CooperBench/cooperdata-bridge-midtrain-blend.