datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
olmo-3-preference-mix-deltas_reasoning-yolo_scottmix-DECON-multi-turnrollouts-olmo32b-rl
rollouts-olmo32b-rl — evaluation rollouts
Model: GRPO ck300 merged from allenai/Olmo-3-1125-32B (adapters: ReasoningRegisters/olmo32b). Tokenizer used for answer positions: allenai/Olmo-3-1125-32B.
Protocol: 32 rollouts per problem (two seeded batches of 16), temperature 0.6, top-p 0.95,
budget 31,744 generated tokens, seed 20260819. Prompts and grader: the paper's repository
(sophicle/reason). Rollout jsonl files are kept as written (one graded rollout per line, with the… See the full description on the dataset page: https://huggingface.co/datasets/reasoning-cues/rollouts-olmo32b-rl.d24-midtrain-olmo3-10b-wholedoc
d24 Midtrain — OLMo-3 Dolmino (10B, whole-doc)
A 9.3B-token reproduction of OLMo 3's Dolma-3 Dolmino mid-train mix, built by
taking a fraction of each component of
allenai/dolma3_dolmino_mix-100B-1025.
No length filter and no chunking — every document is kept whole (some are very long: tens of
thousands of tokens). Each component reaches its target, so the realized mix matches OLMo-3's true
proportions (the OLMo-3 target % column ≈ the kept share). For training, the Megatron… See the full description on the dataset page: https://huggingface.co/datasets/sfanm/d24-midtrain-olmo3-10b-wholedoc.d24-midtrain-olmo3-5b
d24 Midtrain — OLMo-3 Dolmino (5B, chunked)
A 5.00B-token reproduction of OLMo 3's Dolma-3 Dolmino mid-train mix, built by
taking a fraction of each component of
allenai/dolma3_dolmino_mix-100B-1025.
Unlike the smaller d24-midtrain-olmo3
(which dropped documents over 2048 tokens — silently zeroing the long reasoning-trace and PDF
components), this build chunks long documents into 2048-token windows (decoded back to text),
so every component keeps ~100% of its tokens and reaches… See the full description on the dataset page: https://huggingface.co/datasets/sfanm/d24-midtrain-olmo3-5b.LACUNA-data-OLMo3-7B-seed42d24-midtrain-olmo3-5b-wholedoc
d24 Midtrain — OLMo-3 Dolmino (5B, whole-doc)
A 4.71B-token reproduction of OLMo 3's Dolma-3 Dolmino mid-train mix at its exact
component proportions, built by taking a fraction of each component of
allenai/dolma3_dolmino_mix-100B-1025.
Documents are kept whole — no length filter, no chunking.
Why whole-doc: for packed pretrain/midtrain the trainer (e.g. Megatron preprocess_data --append-eod
GPTDataset) already concatenates documents and slices them into context-length windows… See the full description on the dataset page: https://huggingface.co/datasets/sfanm/d24-midtrain-olmo3-5b-wholedoc.dolma3-olmo3-corpus-manifest
dolma3-olmo3-corpus-manifest
Unified per-document manifest variant built against the OLMo3 sidecar schema (1.1B docs, 32-column PyArrow schema with topic + format + quality + token count + source shard).
Provenance
This dataset was renamed on 2026-05-25 as part of the HCAI-Lab HF naming convention cleanup (PR 3). See docs/HCAI_LAB_NAMING_CONVENTION.md in the project repo for the convention.
Field
Value
Previous name
HCAI-Lab/dolma3_olmo3_corpus_manifest… See the full description on the dataset page: https://huggingface.co/datasets/HCAI-Lab-GT/dolma3-olmo3-corpus-manifest.dolma3_mix-150B-1025-merged-olmo3
dolma3_mix-150B-1025-merged (OLMo3)
Tokenized copy of the Dolma3 150B mix (dolma3_mix-150B-1025-merged) using the OLMo3 / Qwen3.5-base (q35base) tokenizer.
Format
Megatron-LM indexed binaries: paired *_text_document.bin and *_text_document.idx files (448 files total, ~613 GiB).
These are not Hugging Face datasets Arrow/Parquet shards. Load them with Megatron / Megatron-LM indexed dataset readers.
Source name
Local / blob name:… See the full description on the dataset page: https://huggingface.co/datasets/yangwang92/dolma3_mix-150B-1025-merged-olmo3.reward-hacking-olmo3.1-32b-kl0.02-seed2-rollouts
Reward-Hacking Training Rollouts — OLMo-3.1-32B (β=0.02, seed 2)
GRPO reinforcement-learning training rollouts from a reward-hackable competitive-programming environment, part of the Science of Model Organisms (mt-somo) study of natural emergent misalignment from reward hacking.
Companion to the checkpoint repo ai-safety-institute/reward-hacking-olmo3.1-32b-kl0.02-seed2. With a small KL penalty (β=0.02) the policy stays closer to the base model, yet it still learns to exploit… See the full description on the dataset page: https://huggingface.co/datasets/ai-safety-institute/reward-hacking-olmo3.1-32b-kl0.02-seed2-rollouts.mmlu_olmo3_contaminationreward-hacking-olmo3.1-32b-kl0.0-seed2-rollouts
Reward-Hacking Training Rollouts — OLMo-3.1-32B (β=0.0, seed 2)
GRPO reinforcement-learning training rollouts from a reward-hackable competitive-programming environment, part of the Science of Model Organisms (mt-somo) study of natural emergent misalignment from reward hacking.
Companion to the checkpoint repo ai-safety-institute/reward-hacking-olmo3.1-32b-kl0.0-seed2. With no KL penalty (β=0) the policy drifts freely from the base model and reliably discovers and exploits the… See the full description on the dataset page: https://huggingface.co/datasets/ai-safety-institute/reward-hacking-olmo3.1-32b-kl0.0-seed2-rollouts.d24-midtrain-olmo3
d24 Midtrain — OLMo-3 Dolmino subsample
A length-filtered (≤2048 GPT-2 tokens) subsample of OLMo 3's Dolma-3 Dolmino mid-train mix,
used for the d24 v1base-olmo3 midtrain. Built by taking a uniform fraction of each
component of allenai/dolma3_dolmino_mix-100B-1025
(so the mix proportions are preserved), then dropping documents >2048 tokens — which naturally
shrinks the long-doc components (reasoning traces, olmOCR PDFs) that don't fit the 2048 context.
5,902,548 documents… See the full description on the dataset page: https://huggingface.co/datasets/sfanm/d24-midtrain-olmo3.FineWeb2-100M-olmo3-7b-toksllm101-olmo3-zh-demo-data
llm001 OLMo3-190M-zh Demo Data
为零基础 AI 大模型研发训练营（llm001）L04 提供的学员数据包。
文件说明
路径
大小
说明
tokenizer/
~4MB
48k BPE 中文分词器，全链路用同一个
nano/tokenized_nano.bin
~1GB
预 tokenize 好的 0.5B tokens（uint16, seed=42 从 3.4B 随机采样 ~15%），直接训
tokenized.bin
~6.8GB
完整 tokenize 数据（3.4B tokens, uint16），有资源可自训完整模型
continue/tokenized_continue.bin
~3.5GB
持续预训练的数据
快速用法（Nano 训练）
from huggingface_hub import snapshot_download
path =… See the full description on the dataset page: https://huggingface.co/datasets/cmz1024/llm101-olmo3-zh-demo-data.olmo-3-preference-mix-deltas_reasoning-chosen_qwen8b-yolo_scottmix-DECONolmo32b-math500-eval-rollouts
olmo32b — MATH-500 eval rollouts
MATH-500 evaluation rollouts for the Olmo-3-32B GRPO run (base allenai/Olmo-3-1125-32B,
RL-Zero prompt, LoRA r=64, 300 steps). Adapters: ReasoningRegisters/olmo32b.
Layout (same as qwen14b-math500-eval-rollouts):
vanilla/step{0,50,100,150,200,250,300}/ — plain prompt, 500 problems × 8 rollouts,
T=0.6, top-p 0.95, 8,192-token cap. step0 = base model; step50+ scored on merged weights.
vanilla/step300_roll32/ — final row: 500 × 32 rollouts, 16… See the full description on the dataset page: https://huggingface.co/datasets/ReasoningRegisters/olmo32b-math500-eval-rollouts.olmo-3-preference-mix-deltas_reasoning-chosen_qwen8b-chat35-DECONolmo-3-preference-mix-deltas_reasoning-yolo_scottmix-chosen_qwen32b_rejected_qwen4b-DECONolmo3-dolma3_mix_baselineolmo-3-preference-mix-deltas-chat0.35_ablate_mathrollouts-olmo32b-screens
rollouts-olmo32b-screens
Model: allenai/Olmo-3-1125-32B (snapshot c2b61dae).
Tokenizer: allenai/Olmo-3-1125-32B.
Protocol: entropy screens: none + the model's top-20 beam nominees, first 30 MATH-train problems x 16 rollouts, budget 16,384, T 0.6, top-p 0.95, seed 20260819; code screens: MBPP beam nominees on HumanEval 164 x 16 (the 32B, on 2 x H200), budget 31,744, execution-graded copies included.
Rollouts generated on the CSAIL cluster for the reasoning-registers paper (Sophie… See the full description on the dataset page: https://huggingface.co/datasets/reasoning-cues/rollouts-olmo32b-screens.Alfworld-Olmo3-7B-Adaptive-Random-Each-evalolmo3-pre-training-qwen3-tokenized-splitaime2024-25-rlvr-olmo3-7b-base-pass64-quartilesrollouts-olmo32b
rollouts-olmo32b — evaluation rollouts
Model: allenai/Olmo-3-1125-32B. Tokenizer used for answer positions: allenai/Olmo-3-1125-32B.
Protocol: 32 rollouts per problem (two seeded batches of 16), temperature 0.6, top-p 0.95,
budget 31,744 generated tokens, seed 20260819. Prompts and grader: the paper's repository
(sophicle/reason). Rollout jsonl files are kept as written (one graded rollout per line, with the
completion), under the run directories as on disk (run/, run_regraded/… See the full description on the dataset page: https://huggingface.co/datasets/reasoning-cues/rollouts-olmo32b.olmo32b-aime-eval-rollouts
olmo32b — AIME 2024/2025 eval rollouts
AIME evaluation rollouts for the Olmo-3-32B GRPO run (adapters:
ReasoningRegisters/olmo32b; MATH-500 rollouts:
ReasoningRegisters/olmo32b-math500-eval-rollouts).
aime2024/step{0,300}/, aime2025/step{0,300}/ — 30 problems × 32 rollouts,
T=0.6, top-p 0.95, 16,384-token cap. step0 = base allenai/Olmo-3-1125-32B,
step300 = merged final checkpoint.
Each directory holds 8 shard_XX.json (per-problem counts) and 8
text_shard_XX.jsonl (one line… See the full description on the dataset page: https://huggingface.co/datasets/ReasoningRegisters/olmo32b-aime-eval-rollouts.olmes-eval-olmo3-7b-instruct-cot
olmes-eval-olmo3-7b-instruct-cot
OLMES evaluation results for OLMo-3-7B instruct with chain-of-thought prompting.
Provenance
This dataset was renamed on 2026-05-25 as part of the HCAI-Lab HF naming convention cleanup (PR 3). See docs/HCAI_LAB_NAMING_CONVENTION.md in the project repo for the convention.
Field
Value
Previous name
HCAI-Lab/data-attribution-olmo-3-7B-instruct-cot-evaluation
Renamed
2026-05-25
See docs/data_home/inventory.json for the… See the full description on the dataset page: https://huggingface.co/datasets/HCAI-Lab-GT/olmes-eval-olmo3-7b-instruct-cot.dclm-replay.seq-4096.n-262144-olmo3
dclm-replay.seq-4096.n-262144-olmo3
OLMo 3-tokenized CPT-replay sequences for prestar, the OLMo analogue of JackHsieh/dclm-replay.seq-4096.tokens-32B (Qwen3).
Source: mlfoundations/dclm-baseline-1.0, pin global-shard_01_of_10/local-shard_0_of_10/*.jsonl.zst.
Tokenizer: allenai/Olmo-3-1025-7B; EOD token id 100257 (<|endoftext|>).
262,144 sequences of exactly 4096 tokens each (docs concatenated and packed; EOD-separated).
Same builder/pin as the Qwen3 replay — corpus is the same… See the full description on the dataset page: https://huggingface.co/datasets/JackHsieh/dclm-replay.seq-4096.n-262144-olmo3.olmes-eval-olmo3-7b-base
olmes-eval-olmo3-7b-base
OLMES evaluation results (per-query predictions + correctness labels) for OLMo-3-7B base on the standard OLMES benchmark suite.
Provenance
This dataset was renamed on 2026-05-25 as part of the HCAI-Lab HF naming convention cleanup (PR 3). See docs/HCAI_LAB_NAMING_CONVENTION.md in the project repo for the convention.
Field
Value
Previous name
HCAI-Lab/data-attribution-olmo-3-7B-base-evaluation
Renamed
2026-05-25
See… See the full description on the dataset page: https://huggingface.co/datasets/HCAI-Lab-GT/olmes-eval-olmo3-7b-base.not-exp-fsid-curated-olmo32b-think-target-100