datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
d24-midtrain-olmo3-10b-wholedoc
d24 Midtrain — OLMo-3 Dolmino (10B, whole-doc)
A 9.3B-token reproduction of OLMo 3's Dolma-3 Dolmino mid-train mix, built by
taking a fraction of each component of
allenai/dolma3_dolmino_mix-100B-1025.
No length filter and no chunking — every document is kept whole (some are very long: tens of
thousands of tokens). Each component reaches its target, so the realized mix matches OLMo-3's true
proportions (the OLMo-3 target % column ≈ the kept share). For training, the Megatron… See the full description on the dataset page: https://huggingface.co/datasets/sfanm/d24-midtrain-olmo3-10b-wholedoc.d24-midtrain-olmo3-5b
d24 Midtrain — OLMo-3 Dolmino (5B, chunked)
A 5.00B-token reproduction of OLMo 3's Dolma-3 Dolmino mid-train mix, built by
taking a fraction of each component of
allenai/dolma3_dolmino_mix-100B-1025.
Unlike the smaller d24-midtrain-olmo3
(which dropped documents over 2048 tokens — silently zeroing the long reasoning-trace and PDF
components), this build chunks long documents into 2048-token windows (decoded back to text),
so every component keeps ~100% of its tokens and reaches… See the full description on the dataset page: https://huggingface.co/datasets/sfanm/d24-midtrain-olmo3-5b.d24-midtrain-olmo3-5b-wholedoc
d24 Midtrain — OLMo-3 Dolmino (5B, whole-doc)
A 4.71B-token reproduction of OLMo 3's Dolma-3 Dolmino mid-train mix at its exact
component proportions, built by taking a fraction of each component of
allenai/dolma3_dolmino_mix-100B-1025.
Documents are kept whole — no length filter, no chunking.
Why whole-doc: for packed pretrain/midtrain the trainer (e.g. Megatron preprocess_data --append-eod
GPTDataset) already concatenates documents and slices them into context-length windows… See the full description on the dataset page: https://huggingface.co/datasets/sfanm/d24-midtrain-olmo3-5b-wholedoc.reward-hacking-olmo3.1-32b-kl0.02-seed2-rollouts
Reward-Hacking Training Rollouts — OLMo-3.1-32B (β=0.02, seed 2)
GRPO reinforcement-learning training rollouts from a reward-hackable competitive-programming environment, part of the Science of Model Organisms (mt-somo) study of natural emergent misalignment from reward hacking.
Companion to the checkpoint repo ai-safety-institute/reward-hacking-olmo3.1-32b-kl0.02-seed2. With a small KL penalty (β=0.02) the policy stays closer to the base model, yet it still learns to exploit… See the full description on the dataset page: https://huggingface.co/datasets/ai-safety-institute/reward-hacking-olmo3.1-32b-kl0.02-seed2-rollouts.reward-hacking-olmo3.1-32b-kl0.0-seed2-rollouts
Reward-Hacking Training Rollouts — OLMo-3.1-32B (β=0.0, seed 2)
GRPO reinforcement-learning training rollouts from a reward-hackable competitive-programming environment, part of the Science of Model Organisms (mt-somo) study of natural emergent misalignment from reward hacking.
Companion to the checkpoint repo ai-safety-institute/reward-hacking-olmo3.1-32b-kl0.0-seed2. With no KL penalty (β=0) the policy drifts freely from the base model and reliably discovers and exploits the… See the full description on the dataset page: https://huggingface.co/datasets/ai-safety-institute/reward-hacking-olmo3.1-32b-kl0.0-seed2-rollouts.d24-midtrain-olmo3
d24 Midtrain — OLMo-3 Dolmino subsample
A length-filtered (≤2048 GPT-2 tokens) subsample of OLMo 3's Dolma-3 Dolmino mid-train mix,
used for the d24 v1base-olmo3 midtrain. Built by taking a uniform fraction of each
component of allenai/dolma3_dolmino_mix-100B-1025
(so the mix proportions are preserved), then dropping documents >2048 tokens — which naturally
shrinks the long-doc components (reasoning traces, olmOCR PDFs) that don't fit the 2048 context.
5,902,548 documents… See the full description on the dataset page: https://huggingface.co/datasets/sfanm/d24-midtrain-olmo3.dclm-replay.seq-4096.n-262144-olmo3
dclm-replay.seq-4096.n-262144-olmo3
OLMo 3-tokenized CPT-replay sequences for prestar, the OLMo analogue of JackHsieh/dclm-replay.seq-4096.tokens-32B (Qwen3).
Source: mlfoundations/dclm-baseline-1.0, pin global-shard_01_of_10/local-shard_0_of_10/*.jsonl.zst.
Tokenizer: allenai/Olmo-3-1025-7B; EOD token id 100257 (<|endoftext|>).
262,144 sequences of exactly 4096 tokens each (docs concatenated and packed; EOD-separated).
Same builder/pin as the Qwen3 replay — corpus is the same… See the full description on the dataset page: https://huggingface.co/datasets/JackHsieh/dclm-replay.seq-4096.n-262144-olmo3.polaris_rose_rollouts_olmo3-7b_from_qwen3-30b-a3b_cutoff4096_240steps
Cross-tokenizer ROSE rollouts — Olmo-3-7B-Think-SFT ← Qwen3-30B-A3B-Thinking-2507
Every assembled row of a complete 240-step online-ROSE run: 61,440 rows, the teacher's
actual continuation for each, and the token accounting behind it.
The student writes a 4096-token prefix in its own vocabulary (100278). That prefix is
decoded to text, the teacher is shown it under its own chat template, and the teacher's
reply comes back as text and is tokenised into the student's vocabulary.… See the full description on the dataset page: https://huggingface.co/datasets/SeanWang0027/polaris_rose_rollouts_olmo3-7b_from_qwen3-30b-a3b_cutoff4096_240steps.olmo3-7b-correctness-contrast-loras
Correctness-contrast LoRAs — Olmo-3-7B
1,972 rank-1 LoRA adapters isolating a causal correctness direction in the weight space of
allenai/Olmo-3-1025-7B, together with that direction, every other direction derived from the
bank, and the full evaluation record on MATH-500 and AIME 2025.
Each adapter is fit on either the correct or the incorrect solutions to a single math problem.
Differencing them cancels the problem and leaves correctness.
Headline. Steering the weights along… See the full description on the dataset page: https://huggingface.co/datasets/amildravid4292/olmo3-7b-correctness-contrast-loras.smolmo-olmo3-calib-4k
Olmo-3 PTQ Calibration Set (4k, math)
A 4,000-sample calibration set for post-training quantization (FP8 / NVFP4) of
allenai/Olmo-3.1-32B-Think. Each row is a
complete math reasoning conversation rendered with the Olmo-3 chat template (the text field),
so calibration sees exactly the model's native inference format — <|im_start|> turn markers, the
Olmo system prompt, <think>…</think> traces, and tool-use scaffolding.
How it was built
Sampled 4 random examples… See the full description on the dataset page: https://huggingface.co/datasets/chankhavu/smolmo-olmo3-calib-4k.olmo-3-7b-think_ifeval
allenai/OLMo-3-7B-Think — ifeval
Model outputs from the micro-creativity inference suite.
Model: allenai/OLMo-3-7B-Think
Dataset: ifeval (541 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 16384
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model (after meta-prompt application)
raw_output
Full… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/olmo-3-7b-think_ifeval.olmo-3-7b-instruct_alpaca-text-generation-384
allenai/OLMo-3-7B-Instruct — alpaca-text-generation-384
Model outputs from the micro-creativity inference suite.
Model: allenai/OLMo-3-7B-Instruct
Dataset: alpaca-text-generation-384 (384 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 16384
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model (after… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/olmo-3-7b-instruct_alpaca-text-generation-384.statML-arxiv-40M-20M-olmo3
statML-arxiv-40M-20M-olmo3
An OLMo 3-tokenized re-windowing of JackHsieh/statML-arxiv-40M-20M
(originally tokenized with Qwen3). Each row is a contiguous span of exactly 4096 tokens under the OLMo 3
tokenizer (allenai/Olmo-3-1025-7B, vocab 100 278), built to be as close as possible to the
corresponding Qwen3 window of the same paper so the two corpora are comparable across tokenizers. The schema
mirrors the ancestor dataset (leading columns), followed by re-windowing/difference… See the full description on the dataset page: https://huggingface.co/datasets/JackHsieh/statML-arxiv-40M-20M-olmo3.olmo-3-7b-think_bookmia-label0-5pct-raw
allenai/OLMo-3-7B-Think — bookmia-label0-5pct-raw
Model outputs from the micro-creativity inference suite.
Model: allenai/OLMo-3-7B-Think
Dataset: bookmia-label0-5pct-raw (247 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 16384
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model (after meta-prompt… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/olmo-3-7b-think_bookmia-label0-5pct-raw.olmo-3-7b-instruct_aime-all
allenai/OLMo-3-7B-Instruct — aime-all
Model outputs from the micro-creativity inference suite.
Model: allenai/OLMo-3-7B-Instruct
Dataset: aime-all (933 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 32768
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model (after meta-prompt application)… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/olmo-3-7b-instruct_aime-all.olmo-3-7b-instruct_tinystories-val1pct-raw
allenai/OLMo-3-7B-Instruct — tinystories-val1pct-raw
Model outputs from the micro-creativity inference suite.
Model: allenai/OLMo-3-7B-Instruct
Dataset: tinystories-val1pct-raw (220 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 16384
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model (after… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/olmo-3-7b-instruct_tinystories-val1pct-raw.olmo-3-7b-think_alpaca-text-generation-384
allenai/OLMo-3-7B-Think — alpaca-text-generation-384
Model outputs from the micro-creativity inference suite.
Model: allenai/OLMo-3-7B-Think
Dataset: alpaca-text-generation-384 (384 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 16384
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model (after… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/olmo-3-7b-think_alpaca-text-generation-384.olmo-3-7b-think_aime-all
allenai/OLMo-3-7B-Think — aime-all
Model outputs from the micro-creativity inference suite.
Model: allenai/OLMo-3-7B-Think
Dataset: aime-all (933 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 32768
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model (after meta-prompt application)
raw_output
Full… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/olmo-3-7b-think_aime-all.olmo-3-7b-instruct_bookmia-label0-5pct-raw
allenai/OLMo-3-7B-Instruct — bookmia-label0-5pct-raw
Model outputs from the micro-creativity inference suite.
Model: allenai/OLMo-3-7B-Instruct
Dataset: bookmia-label0-5pct-raw (247 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 16384
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model (after… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/olmo-3-7b-instruct_bookmia-label0-5pct-raw.olmo-3-7b-think_creativemath-with-answers
allenai/OLMo-3-7B-Think — creativemath-with-answers
Model outputs from the micro-creativity inference suite.
Model: allenai/OLMo-3-7B-Think
Dataset: creativemath-with-answers (188 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 32768
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model (after… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/olmo-3-7b-think_creativemath-with-answers.olmo-3-7b-instruct_creativemath-with-answers
allenai/OLMo-3-7B-Instruct — creativemath-with-answers
Model outputs from the micro-creativity inference suite.
Model: allenai/OLMo-3-7B-Instruct
Dataset: creativemath-with-answers (188 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 32768
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model (after… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/olmo-3-7b-instruct_creativemath-with-answers.olmo-3-7b-instruct_ifeval
allenai/OLMo-3-7B-Instruct — ifeval
Model outputs from the micro-creativity inference suite.
Model: allenai/OLMo-3-7B-Instruct
Dataset: ifeval (541 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 16384
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model (after meta-prompt application)
raw_output… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/olmo-3-7b-instruct_ifeval.olmo-3-7b-instruct_writingbench-en100
allenai/OLMo-3-7B-Instruct — writingbench-en100
Model outputs from the micro-creativity inference suite.
Model: allenai/OLMo-3-7B-Instruct
Dataset: writingbench-en100 (100 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 8192
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model (after meta-prompt… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/olmo-3-7b-instruct_writingbench-en100.olmo-3-7b-think_arena-hard-creative-writing
allenai/OLMo-3-7B-Think — arena-hard-creative-writing
Model outputs from the micro-creativity inference suite.
Model: allenai/OLMo-3-7B-Think
Dataset: arena-hard-creative-writing (250 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 16384
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model (after… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/olmo-3-7b-think_arena-hard-creative-writing.olmo-3-7b-think_storygen-prompts-200
allenai/OLMo-3-7B-Think — storygen-prompts-200
Model outputs from the micro-creativity inference suite.
Model: allenai/OLMo-3-7B-Think
Dataset: storygen-prompts-200 (200 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 16384
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model (after meta-prompt… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/olmo-3-7b-think_storygen-prompts-200.olmo-3-7b-instruct_storygen-prompts-200
allenai/OLMo-3-7B-Instruct — storygen-prompts-200
Model outputs from the micro-creativity inference suite.
Model: allenai/OLMo-3-7B-Instruct
Dataset: storygen-prompts-200 (200 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 16384
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model (after meta-prompt… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/olmo-3-7b-instruct_storygen-prompts-200.olmo-3-7b-instruct_arena-hard-creative-writing
allenai/OLMo-3-7B-Instruct — arena-hard-creative-writing
Model outputs from the micro-creativity inference suite.
Model: allenai/OLMo-3-7B-Instruct
Dataset: arena-hard-creative-writing (250 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 16384
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/olmo-3-7b-instruct_arena-hard-creative-writing.olmo-3-7b-think_tinystories-val1pct-raw
allenai/OLMo-3-7B-Think — tinystories-val1pct-raw
Model outputs from the micro-creativity inference suite.
Model: allenai/OLMo-3-7B-Think
Dataset: tinystories-val1pct-raw (220 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 16384
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model (after meta-prompt… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/olmo-3-7b-think_tinystories-val1pct-raw.olmo3-190m-zh-v2-base-data
OLMo3-190M-zh-v2 Base Tokenized Data
这是 OLMo3-190M 中文 v2 base pretrain 使用的正式 tokenized 数据。
本仓库只发布 packed token id,不发布 raw text shards。这样可以让训练复现直接读取 tokenized.bin,同时避免原始语料再分发带来的体积和授权边界问题。
文件说明
tokenized.bin # uint16 一维 token 流
meta.json # 数据构建、混合比例、token 统计等元数据
必须匹配的 Tokenizer
本数据必须使用下面这个 tokenizer 解码和训练:
Nwna/olmo3-190m-zh-v2-tokenizer
不要用其他 tokenizer 读取这份 tokenized.bin。同一个数字 token id 在不同 tokenizer 里含义不同,混用会导致训练目标错位,即使 loss 下降也可能训练出坏模型。
数据摘要… See the full description on the dataset page: https://huggingface.co/datasets/Nwna/olmo3-190m-zh-v2-base-data.olmo-3-7b-think_writingbench-en100
allenai/OLMo-3-7B-Think — writingbench-en100
Model outputs from the micro-creativity inference suite.
Model: allenai/OLMo-3-7B-Think
Dataset: writingbench-en100 (100 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 8192
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model (after meta-prompt… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/olmo-3-7b-think_writingbench-en100.
