datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
fineweb-edu-gemma4-1024
FineWeb-Edu — pre-tokenized for fast LM pretraining (Gemma tokenizer, ArrayRecord/Grain)
Pre-tokenized FineWeb-Edu
(sample/100BT), packed into fixed-length sequences and stored as
ArrayRecord shards for zero-overhead
streaming with Grain. No on-the-fly tokenization
at train time — you read int32 tokens straight off disk.
Format
Tokenizer: google/gemma-4-12B-it (vocab size 262144). Documents are
separated by the EOS token id 1.
Packing: the token stream is… See the full description on the dataset page: https://huggingface.co/datasets/mlnomad/fineweb-edu-gemma4-1024.gemma4-e2b-base-topk128-hf-overlay-v128-seed42
Gemma 4 E2B base top-k-128 HF training overlay
This is the immutable training-engine overlay used to distill traces from Gemma 4 E2B base into
Gemma 4 E4B. It preserves the prompts, responses, and exact response token IDs from
JWei05/gemma4-e2b-base-topk128-traces,
but replaces the source vLLM top-k targets with targets recomputed by the Hugging Face training
engine.
This repository is a reproducibility artifact for the corresponding distillation run. It is not a
new… See the full description on the dataset page: https://huggingface.co/datasets/JWei05/gemma4-e2b-base-topk128-hf-overlay-v128-seed42.gemma4-german-tutor-data
German Tutor — grammar correction, conversation & flashcard data
The training set, evaluation suites, source lexicons and eval results behind
kessenma/gemma4-e4b-german-tutor-4bit
— a Gemma 4 E4B fine-tune that runs fully on-device (MLX, 4-bit) as the tutor in a German
learning app.
The fine-tune lifted the core grammar suite from 72% → 85%, halved missed errors
(17% → 9%), and cut false corrections (34% → 22%). Everything needed to reproduce those
numbers is in this repo.… See the full description on the dataset page: https://huggingface.co/datasets/kessenma/gemma4-german-tutor-data.cot-gemma4-26b-a4b
Gemma-4-26B-A4B-it Chain-of-Thought Oracle Corpus
Chain-of-thought rollouts generated with google/gemma-4-26B-A4B-it (MoE,
25.2B total / 3.8B active), in its native thinking mode, across a diverse suite
of reasoning tasks. Structure follows
ceselder/cot-oracle-corpus-v5
(CoT-only subset of the columns), built for chain-of-thought monitoring /
activation-oracle research.
2,121,354 rollouts over 212,161 unique problems (10 sampled
thinking rollouts per problem, temperature 0.8).… See the full description on the dataset page: https://huggingface.co/datasets/cds-jb/cot-gemma4-26b-a4b.synthweb-gemma4-26b-a4b
Gemma-4-26B-A4B FineWeb Rollouts (~580k docs)
Open-ended continuations of FineWeb
(sample-10BT) document prefixes, generated by google/gemma-4-26b-a4b (the base, non-it
Gemma-4 26B-A4B mixture-of-experts model), then mode-collapse filtered. This is the Gemma-4
analogue of cds-jb/qwen3-8b-fineweb-rollouts-100k:
a "synthweb" corpus of natural model-generated documents, intended as the substrate for
activation-oracle / interpretability probing (extract a base model's residual… See the full description on the dataset page: https://huggingface.co/datasets/cds-jb/synthweb-gemma4-26b-a4b.gemma4-german-sft-corpus
Gemma-4-E4B German SFT Corpus — 4 controlled variants
Curated, native-heavy German supervised-fine-tuning (SFT) corpus, built to improve the
general German skill of unsloth/gemma-4-E4B-it via LoRA — NOT to target any single
benchmark. The EuroEval-ported German benchmarks (scala_de, sb10k_de, include_de,
mmlu_prox_de, germeval_de, germanquad_de, …) are used only as honest thermometers, never
as training signal — no benchmark train/test split is mixed in, deliberately, to avoid… See the full description on the dataset page: https://huggingface.co/datasets/peerbench/gemma4-german-sft-corpus.gemma4-code-review-instruct
gemma4-code-review-instruct
197K code review examples — 58K with chain-of-thought <think> reasoning traces.
Built to train models that don't just flag issues, but explain their reasoning before delivering a review. Drop-in ready for SFT with any chat model.
Why This Dataset
Most code review datasets give you diff → comment. This one gives you diff → think → comment for 30% of examples — reasoning traces that show how to analyze a diff before writing the review.… See the full description on the dataset page: https://huggingface.co/datasets/liodon-ai/gemma4-code-review-instruct.gemma4-mtp-quality-study
Gemma 4 MTP Quality–Throughput Study
Author: Taiko Toeda, Independent ResearcherORCID: https://orcid.org/0009-0001-7267-0201Version: 1.0.0DOI: 10.5281/zenodo.21860461
This Dataset repository distributes the sanitized processed measurements from
an independent paired benchmark of QAT Q4_0 Gemma 4 12B IT and 26B-A4B IT on
one RTX 5070 Ti using pinned CUDA llama.cpp commit
7ba604f1cb61cd14898138e9abc0b4ff2601f180.
Included data
request-level timing, throughput… See the full description on the dataset page: https://huggingface.co/datasets/moebiusT7/gemma4-mtp-quality-study.gemma4-31b-tool-selector-sft-v1.1
Gemma 4 31B Tool Selector SFT v1.1
Balanced supervision for a strict single-call selector that either emits one
supported deterministic tool invocation or explicitly defers to a fixed neural
verifier. This is the training lineage for the selected Gemma 4 31B selector
adapter.
Contents
Split
Rows
Tool
Defer
Purpose
train
1,408
704
704
Optimization
validation
384
192
192
Training-time validation
audit
256
—
—
Final audit only
Both canonical… See the full description on the dataset page: https://huggingface.co/datasets/Kobarac/gemma4-31b-tool-selector-sft-v1.1.2026_08_26_omni_math_train_feedback_adherence_gemma3_12b_gemma4_31b_candidates
Omni-MATH train feedback-adherence candidates
Production candidate data for studying whether a student follows teacher feedback.
Student: google/gemma-3-12b-it
Teacher and adherence judge: google/gemma-4-31B-it
Source problems: LLParallax/Omni-MATH-filtered, train partition after a fixed 512-problem test split
Source trajectories: LLParallax/2026_07_16_collect_omni_math_gemma3_12b_gemma4_31b
Collection config:… See the full description on the dataset page: https://huggingface.co/datasets/1337xyz1337xyz/2026_08_26_omni_math_train_feedback_adherence_gemma3_12b_gemma4_31b_candidates.nemotron-cc-atomic-simplification-gemma4-31b
nemotron-cc atomic-statement simplification (Gemma 4 31B-it)
2,000,000 records: source text from
nvidia/nemotron-cc-v2.1 (High-Quality-Synthetic
split) rewritten by google/gemma-4-31B-it into a sequence of atomic, Subject-Verb-Object
statements.
Generated with vLLM 0.22.1 in-process batch inference (see src/generate/run.py in the
producing repo), TP=4, max_model_len=16384, max_tokens=8192, prompts filtered to
<=8192 templated tokens.
Fields
id: original… See the full description on the dataset page: https://huggingface.co/datasets/rpisano/nemotron-cc-atomic-simplification-gemma4-31b.ocn-empty-negations-generations-main-gemma4-qwen35
OCN OSS Model Generations
This dataset contains open-source model generations for prompts designed to elicit or suppress contrastive-negation framing.
Columns
prompt metadata from the OCN prompt bank;
model_id: Hugging Face model id;
model_family: model family;
model_stage: base, instruct, or other;
decoding: decoding configuration name;
seed: generation seed;
response: generated answer;
created_at: notebook run timestamp.
experiment_id: experiment cohort… See the full description on the dataset page: https://huggingface.co/datasets/ritwikraha/ocn-empty-negations-generations-main-gemma4-qwen35.gemma4-onpolicy-student-corrections
Gemma 4 12B FrontierDistill - On-Policy Student Failure Corrections
Attribution Requirement: This dataset was created and curated by True2456. Any use, redistribution, derivative dataset, model fine-tune, or paper using this dataset MUST cite and reference True2456 and the Gemma 4 12B FrontierDistill Project.
This dataset contains 2,000 on-policy student failure corrections collected live from Gemma 4 12B (gemma-4-12b-it-qat-frontierdistill).
Every example in this dataset… See the full description on the dataset page: https://huggingface.co/datasets/True2456/gemma4-onpolicy-student-corrections.gemma4-multimodal-recipe-dataset
🍳 Gemma 4 Multimodal Recipe & Food Dataset
A balanced, high-density multimodal dataset curated specifically for fine-tuning compact vision-language models (such as gemma-4-e2b-it) for visual food recognition, recipe generation, and dietary recommendation.
🔗 Upstream & Source Datasets
This dataset was created by cleaning, reformatting, and synthesizing samples across the following 5 Hugging Face sources:
Dataset
Modality
Role in Pipeline… See the full description on the dataset page: https://huggingface.co/datasets/alst10/gemma4-multimodal-recipe-dataset.Gemma4-Terminal-ToolBench-Tokenized
Gemma4 Terminal ToolBench Tokenized Datasets
Gemma4-family tokenized terminal/tool datasets for HF/FSDP experiments.
Contents
gemma4_e2b_hf_fsdp_1024: 23472 rows, 0.28 GiB, features: input_ids, attention_mask, labels
gemma4_e2b_hf_fsdp_1024_4gpu: 23472 rows, 0.28 GiB, features: input_ids, attention_mask, labels
gemma4_e4b_hf_fsdp_1024: 23472 rows, 0.28 GiB, features: input_ids, attention_mask, labels
gemma4_26b_a4b_hf_fsdp_1024: 23472 rows, 0.28 GiB, features:… See the full description on the dataset page: https://huggingface.co/datasets/LLM-OS-Models/Gemma4-Terminal-ToolBench-Tokenized.caliber-extension-gemma4-e2b-grpo-rollouts
CALIBER Extension — Gemma4-E2B GRPO Rollouts
Training rollouts from matched GRPO arms on google/gemma-4-E2B-it
(new-prompt template, non-thinking, full bf16, max completion 1500, 150 steps).
Subsets
subset
arm
τ
prior
rows
mean reward_total
accuracy
full schema
caliber
vanilla CALIBER
0.0
—
1600
2.298
0.514
0.664
mink
Min-K% prior
1.0
mink_0.2
4800
2.506
0.520
0.680
minkpp
Min-K++% prior
1.0
minkpp_0.2
4800
2.637
0.541
0.726
Load:
from datasets… See the full description on the dataset page: https://huggingface.co/datasets/dmnsh/caliber-extension-gemma4-e2b-grpo-rollouts.cyberforge-teacher-traj-gemma4-31b
CyberForge Teacher Trajectories (Gemma-4-31B)
880 agentic security-patch trajectories from the Gemma-4-31B self-distillation teacher, cleansed to the
final versions used to train the student models in the CyberForge paper.
Each line is one trajectory (JSONL): messages (system / user / assistant turns of the
mini-swe-agent loop) and metadata.
Teacher: Gemma-4-31B self-distillation teacher
Records: 880
Format: JSONL, one trajectory per line
Related
Companion… See the full description on the dataset page: https://huggingface.co/datasets/AmL-hug/cyberforge-teacher-traj-gemma4-31b.nla-gemma4e2b-relabel-v1-eval
Gemma-4-E2B layer-23 evaluation set, relabeled (v1), with contamination flags
The 580-document evaluation pool on which every activation-verbalizer result in this
project is scored, with each row's evaluation text rewritten from a topic summary to
a feature-attribution label. The activations are byte-identical to the original
evaluation set; only the text column changed, and the original text is preserved.
This pool is not disjoint from the training corpus. Read this… See the full description on the dataset page: https://huggingface.co/datasets/Solshine/nla-gemma4e2b-relabel-v1-eval.gemma4-onpolicy-50topics-2000-corrections
Gemma 4 12B FrontierDistill - 2,000 Authentic 50-Topics On-Policy Student Failure Corrections
Attribution Requirement: This dataset was created and curated by True2456. Any use, redistribution, derivative dataset, model fine-tune, or paper using this dataset MUST cite and reference True2456 and the Gemma 4 12B FrontierDistill Project.
This dataset contains 2,000 authentic on-policy student failure corrections collected live from Gemma 4 12B (gemma-4-12b-it-qat-frontierdistill)… See the full description on the dataset page: https://huggingface.co/datasets/True2456/gemma4-onpolicy-50topics-2000-corrections.gemma4-agent-sft
gemma4-agent-sft
A clean, deduplicated, mixture-balanced tool-calling agent SFT dataset for fine-tuning google/gemma-4-26B-A4B-it, normalized from three agentic sources (Agent-Ark/Toucan-1.5M, open-thoughts/AgentTrove, nvidia/Nemotron-SFT-Agentic-v2).
Format (text, not pre-tokenized)
default config — 132,909 examples. Columns:
id, source, source_subset, tool_names (list)
messages — JSON string: list of {role, content, tool_calls, tool_responses}
tools — JSON… See the full description on the dataset page: https://huggingface.co/datasets/voidful/gemma4-agent-sft.gemma-4-31b-it_writingbench-en100
google/gemma-4-31b-it — writingbench-en100
Model outputs from the micro-creativity inference suite.
Model: google/gemma-4-31b-it
Dataset: writingbench-en100 (100 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 8192
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model (after meta-prompt application)… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/gemma-4-31b-it_writingbench-en100.nla-gemma4e2b-relabel-v1-corpus
Gemma-4-E2B layer-23 activation corpus, relabeled (v1)
1356 training rows for an activation verbalizer. Each row pairs a residual-stream
activation captured at layer 23 of google/gemma-4-E2B with a natural-language label
describing what the model must have integrated at that position to predict its next
token. This is the training set behind
Solshine/gemma-4-e2b-nla-L23-av-priordev-relabel-v1-wd3.
Why it exists
An audit of the previous version of this corpus found… See the full description on the dataset page: https://huggingface.co/datasets/Solshine/nla-gemma4e2b-relabel-v1-corpus.gemma4-opus-reasoning-12k
Opus 4.6 Reasoning SFT 12k
A unified, pre-cleaned reasoning dataset built from 4 Claude Opus 4.6 distillation sources. Ready for supervised fine-tuning — just load and train.
Why This Dataset Exists
The source datasets have different schemas, null values, and reasoning stored in non-standard keys that apply_chat_template() silently drops. This dataset fixes all of that:
Reasoning traces merged into assistant content using <think>...</think> tags
Null/empty content… See the full description on the dataset page: https://huggingface.co/datasets/Farseen0/gemma4-opus-reasoning-12k.gemma4-opus-reasoning-12k
Opus 4.6 Reasoning SFT 12k
A unified, pre-cleaned reasoning dataset built from 4 Claude Opus 4.6 distillation sources. Ready for supervised fine-tuning — just load and train.
Why This Dataset Exists
The source datasets have different schemas, null values, and reasoning stored in non-standard keys that apply_chat_template() silently drops. This dataset fixes all of that:
Reasoning traces merged into assistant content using <think>...</think> tags
Null/empty content… See the full description on the dataset page: https://huggingface.co/datasets/rahul7star/gemma4-opus-reasoning-12k.gemma4-qwen35-gsm8k-rollouts
Gemma 4 and Qwen3.5 GSM8K Rollouts
This dataset contains 3,957 saved generations from three complete runs over
the 1,319-example openai/gsm8k main test split:
Model
Rows
Strict match
Flexible extract
google/gemma-4-26B-A4B
1,319
33.28%
39.95%
google/gemma-4-E4B
1,319
26.23%
30.86%
Qwen/Qwen3.5-35B-A3B
1,319
15.92%
23.12%
Every row includes the exact five-shot prompt, model generation, reference
answer, strict and flexible correctness flags, pinned… See the full description on the dataset page: https://huggingface.co/datasets/dureduck/gemma4-qwen35-gsm8k-rollouts.personahub-teacher-scale-9k-gemma4-sft-20260514
PersonaHub Teacher Scale 9k Gemma4 SFT
Trainer-ready JSONL for TRL/Gemma SFT. Each row has messages, and the final message is the assistant target.
This is an emergency synthetic PersonaHub-seeded SFT pilot artifact generated on 2026-05-14. Treat as research/training pilot data; run qualitative audits before production training decisions.
Files:
train.jsonl: trainer-ready messages format
manifest.json: counts and provenance summary
Schema per row:
{"case_id":"..."… See the full description on the dataset page: https://huggingface.co/datasets/Pranavz/personahub-teacher-scale-9k-gemma4-sft-20260514.Gemma-4-E2B-SSFT
Gemma-4-E2B-SSFT
This is a dataset that has been created using SSFT (simple-sft), a synthetic data generation tool written by me.
It contains a few hundred samples for testing.
Try it out yourself!
gemma4-rhyme-interp
Gemma-4 Rhyme Interpretability — datasets
Evaluation and probing datasets from a mechanistic interpretability study of how
google/gemma-4-E2B (base) completes the last word of a rhyming line. The full
analysis, code, and write-ups (reports 01–10, including the circuit, the
localization of the rhyme "write" to a single MLP, the value-memory readout, and
a training-free rank-1 weight edit that installs a false rhyme) live in the
GitHub repository:… See the full description on the dataset page: https://huggingface.co/datasets/eac123/gemma4-rhyme-interp.gemma-4-31B-korean-on-policy-150k
Korean On-Policy QA (Gemma 4) — EAGLE-3 training data
Korean instruction/response pairs where the responses were regenerated on-policy by
a Gemma 4 verifier. Originally built to retrain an EAGLE-3 speculator for Korean, but
also usable for general Korean instruction-tuning / distillation.
Structure
Rows: ~150,000
Language: Korean
Columns:
instruction (string) — the question / instruction
output (string) — the verifier-generated response
messages (list) — chat… See the full description on the dataset page: https://huggingface.co/datasets/BCCard/gemma-4-31B-korean-on-policy-150k.gemma-4-e2b-deception-behavior-completions
Gemma-4-E2B deception & behavior completions
Consolidated 910-row corpus of (scenario prompt + Gemma-4-E2B-generated completion) pairs from earlier mechanistic-interpretability experiments. Each row captures the prompt the model saw and the text it actually produced; for a subset, Claude-Haiku-4-5 judge verdicts and SAE-feature labels are included.
The corpus is meant to be used as activation-extraction input for downstream interpretability work — Natural Language Autoencoder (NLA)… See the full description on the dataset page: https://huggingface.co/datasets/Solshine/gemma-4-e2b-deception-behavior-completions.