mihailgribov/olympiad_style_integer_math_problems

Olympiad Math Corpus

Version: v2.1.1<br> Release date: 2026-05-03

59,486 synthetically generated olympiad-style math problems with verified integer answers and formal computation graphs.

Loading

from datasets import load_dataset

ds = load_dataset("mihailgribov/olympiad_style_integer_math_problems", split="train")

lemma_applicability is stored as list[{lemma, status}] rather than a sparse dict (required for Arrow-based consumers). To convert to a dict for local use:

lemma_app = {x["lemma"]: x["status"] for x in row["lemma_applicability"]}

Overview

Each problem is generated from a computation graph (CG-Python DSL) that formally defines the mathematical structure and answer. Problem text is produced by an LLM from the graph. Every answer is verified by independent evaluation of the graph by a deterministic evaluator.

Domains are classified by graph structure, not by problem text.

Files

The release ships the full dataset in both Parquet and JSONL formats. load_dataset and the HuggingFace viewer use ready.parquet; ready.jsonl is provided for direct reading, grep, and streaming without the datasets library. Both files contain the same records.

ready_sample_50.jsonl contains 50 problems sampled stratified by olympiad_level: allocations follow the dataset's OL distribution proportionally (largest-remainder method), with at least one problem from every non-empty OL. Train/validation/test splits are intentionally left to the consumer — the dataset ships as a single shuffled file.

Problem Schema (ready.parquet / ready.jsonl, identical)

<details> <summary>Example record</summary>

{
  "id": "12be79",
  "alias": "nt_sum_divisors_mod_v1_124444284_4622",
  "problem": "Let $n$ be the number of positive integers less than or equal to $674$ that are relatively prime to $15$. Let $\\sigma$ be the sum of the positive divisors function. Compute $\\sigma(n) \\pmod{89}$.",
  "answer": 1170,
  "graph": "graphs = [\\n    Graph(\\n        let={\\n            \"n\": CountOverSet(...)\\n        },\\n        goal=\"result\"\\n    )\\n]",
  "domain": "NT",
  "secondary_domain": null,
  "goal": "COMPUTE",
  "evaluator_id": "sympy",
  "root_lemma": "C4",
  "lemma_paths": ["C4"],
  "recipe_id": "08d162",
  "seed_template_id": "nt_sum_divisors_mod_v1",
  "ending_id": null,
  "olympiad_level": 5,
  "num_spawns": 0,
  "lemma_set": ["C4"],
  "num_lemmas": 1,
  "generation_time": 0.001,
  "created_at": "2026-02-08T06:06:42.273144Z",
  "verification": {
    "verified": true,
    "answer": 1170,
    "timestamp": "2026-02-08T06:06:42.274633Z"
  },
  "problem_hash": "49f89c",
  "license": "CC BY 4.0",
  "llm_solvers": [
    {
      "id": 5,
      "model": "deepseek-ai/DeepSeek-V3.2",
      "answer": 1170,
      "score": 3,
      "correct": {"strict": true, "boxed": true, "relaxed": true},
      "usage": {"prompt_tokens": 102, "completion_tokens": 828},
      "timestamp": "2026-02-12T20:35:01.853Z"
    }
  ],
  "solution_status": 1,
  "lemma_applicability": [
    {"lemma": "C4", "status": "ok"},
    {"lemma": "K17", "status": "no"},
    {"lemma": "L3b", "status": "no"},
    {"lemma": "V3", "status": "no"}
  ],
  "irt_difficulty": {"lo": -10.0, "mid": -3.35, "hi": 4.18}
}

</details>

verification

lemma_applicability

List of {lemma, status} entries indicating whether each candidate lemma is the correct first solving step. Sorted by lemma for determinism. Empty list when lemma_paths is empty. Non-empty on 56,949 of 59,486 problems.

"lemma_applicability": [
  {"lemma": "K3", "status": "ok_later"},
  {"lemma": "K5", "status": "same_pattern_wrong"},
  {"lemma": "V1", "status": "ok"},
  {"lemma": "V7", "status": "no"}
]

Lemma Catalog (lemmas_used.jsonl)

The file contains 88 mathematical lemmas (identities and reduction rules) that appear in at least one problem in the dataset. Each lemma is a named transformation used by the symbolic solver to reduce computation graphs. Lemma IDs are referenced by the lemma_set, lemma_paths, root_lemma, and lemma_applicability fields in problem records.

<details> <summary>Example record</summary>

{
  "id": "LIN_FORM",
  "name": "linear_form_range_counting",
  "type": "solver_lemma",
  "domains": ["algebra"],
  "level": "olympiad",
  "track": "olympiad_nt",
  "description": "count of linear form values.",
  "description_latex": "|\\{x \\in [a,b] : \\exists\\, k,\\; x = \\alpha + k\\beta\\}|",
  "dataset_count": 15888,
  "dataset_fraction": 0.1951,
  "dataset_as_root": 13176
}

</details>

Statistics

Olympiad Level

Mathematical sophistication level assigned by GPT-5.1 from the computation graph alone (no problem text). Measures the rarity and non-obviousness of the required mathematical insight. This is distinct from solve difficulty: the correlation with empirical irt_difficulty is weak (Pearson r ≈ 0.23). Problems scoring below 2 are excluded from the dataset as trivial or invalid.

<details> <summary>Scoring prompt used for <code>olympiad_level</code></summary>

The label is the output of a single, deterministic LLM call. The system prompt (reproduced verbatim below) fixes the evaluation axis so scores are comparable across releases.

You are a mathematical olympiad problem evaluator.

You receive computation graphs written in a Python DSL (CG-Python).
You do NOT see the natural-language problem statement.

The `goal` node is the value to compute.
`Ref("x")` denotes a previously defined variable.
`Var("x")` denotes a bound variable inside set comprehensions.

Your task is to assess the olympiad level of each problem on a scale 0–9.

CORE PRINCIPLE
    Grade the problem by the mathematical insight that is logically required
    to solve it, as inferred from the graph structure.
    Do NOT reward graph size, depth, nesting, large constants,
    or routine mechanical transformations.
    Key question: What mathematical idea is necessary here,
    and how obvious is it?

GRAPH-ONLY EVALUATION
    Infer difficulty ONLY from the graph.
    - which mathematical concepts are involved,
    - whether the goal requires a single known theorem directly,
      or a non-obvious connection between concepts,
    - whether quantifiers, sets, or extrema introduce real reasoning
      rather than simple wrapping.
    Do NOT treat the mere presence of advanced-looking nodes
    (Factorial, Binomial, EulerPhi, MoebiusMu, Lucas, etc.)
    or deep nesting as insight.

LEVELS (0–9)
    0 — Invalid / non-interpretable graph.
    1 — Bare textbook definition lookup.
    2 — Trivial; one-line fact.
    3 — Direct application of a single theorem.
    4 — Exam-style; correct method is obvious.
    5 — Training olympiad; standard idea, execution-heavy.
    6 — School olympiad; exactly one non-obvious key insight.
    7 — Regional / strong olympiad; combine standard ideas non-trivially.
    8 — National / IMO-shortlist; requires seeing hidden structure.
    9 — Top international olympiad; genuine non-routine insight or
        construction, not reducible to recalling a named theorem.

COMBINATION RULE
    If solving the graph requires combining multiple standard ideas
    in a way that is not obvious from the graph alone, increase the
    level by +1 relative to using any one idea alone.
    Do NOT increase the level if ideas are applied only sequentially
    or mechanically.

WRAPPER PENALTY
    If a complex inner subgraph merely computes a known invariant
    and the outer computation is routine arithmetic, score the problem
    by the outer goal only. Wrapping a known result inside a trivial
    shell does NOT increase the level. Such problems typically remain
    level 2–4.

OUTPUT FORMAT
    Output TSV, one line per graph:   id<TAB>level
    No explanations, no headers, no code fences.

The user message is a concatenation of the graph strings, one per target problem. Temperature and other decoding parameters match the text_gen pipeline configuration.

</details>

Scoring rules: (1) wrapping a known result in trivial arithmetic does not increase the level; (2) combining multiple independent ideas non-trivially adds +1; (3) graph size, nesting depth, and large constants are not rewarded.

IRT Difficulty

Calibrated difficulty estimate based on Item Response Theory (IRT-1PL / Rasch model). Present for 59,486 tasks attempted by at least one LLM solver.

The Rasch model defines the probability that model j solves task i as:

P(correct) = σ(θⱼ − βᵢ)

where σ is the logistic function, θⱼ is model skill, and βᵢ is task difficulty. Higher β = harder task. Model skills (θ) are estimated jointly on "core" tasks attempted by ≥3 models; task difficulties (β) are then estimated per-task with fixed θ.

"irt_difficulty": {"lo": -1.86, "mid": 2.91, "hi": 7.44}

Confidence intervals are computed via profile likelihood: the set of β values where the log-likelihood is within χ²(1)/2 = 1.92 of the maximum. This gives finite, interpretable bounds even for tasks attempted by a single model, where Wald-type intervals (β ± 1.96·SE) would diverge. The interval width reflects estimation uncertainty: tasks attempted by more models have narrower intervals (typical width 4–6) than single-model tasks (width 15–18).

Answer Distribution

All answers are non-negative integers in [0, 99999]. The answer distribution is non-uniform but smooth, with full-range coverage and no abrupt discontinuities.

The median answer is 11,741 and the mean is 1,482,189. The distribution is concentrated in the lower range (47.9% of answers fall in [0, 9,999]), reflecting the typical output magnitude of small integer-valued olympiad problems, but has smooth coverage across the full [0, 99,999] range without gaps. Very small values such as 0, 1, and 2 do not dominate the distribution and occur with comparable, low frequencies, so no special or degenerate cases collapse into these values.

LLM Solvers

Each model receives the problem text as the user message with the following system prompt:

You are a participant of the International Mathematical Olympiad. Solve the given problem. Think step by step. Write your final answer as a single integer inside \boxed{}, for example: \boxed{42}

All models are queried with temperature=0 and max_completion_tokens=32768. No few-shot examples or chain-of-thought scaffolding is provided beyond the system prompt above.

¹ Solve rate is not a comparable measure of model capability across rows of this table. Each solver was run on a different subset of problems (cost, rate limits, and the pilot vs. continuous phase of a given model all shaped the attempt budget), so the "Attempted" column varies by more than 10× between models and the per-row solve rate is computed against a different task pool for each row. For capability comparisons that account for task difficulty, use irt_difficulty on the task side together with jointly-fit model skill (θ).

For a per-OL view of how each model degrades with task difficulty, see the figure below. The 9 solvers are ordered on a rough capability ladder (top-left = strongest, bottom-right = weakest). For each model, attempts are grouped by olympiad_level (OL=2..9) and stacked: green = correct.strict (last \boxed{} matches the expected answer), red = wrong. Cells with fewer than 30 attempts are faded (not statistically significant).

[image]

Overall solver coverage:

• —Total solver attempts: 252,773
• —Total correct (strict): 117,945
• —Tasks with ≥1 correct solution: 58,634 (98.6%)
• —Mean attempts / task: 4.25
• —Mean correct solutions / task: 1.98

Each llm_solvers entry:

Correctness levels (each level implies the ones below it):

Solution Status

Aggregate LLM correctness per problem. Derived from individual llm_solvers entries. Only problems attempted by at least one LLM solver receive a status; the rest are null.

Novelty

Problems are constructed from deterministic symbolic structures without using existing problem texts; large language models are used only for natural-language rendering after the mathematical content is fixed. This avoids reuse of web-circulated problems or content memorized by language models.

Curation and Quality Control

The dataset is produced by an automated generation pipeline followed by filtering. The following checks are applied to improve the overall quality of the dataset.

• —Seed verification. Each problem starts from a simple seed graph that is directly solvable by a symbolic engine (e.g. SymPy, Google OR-Tools). Before any further transformations, the seed is evaluated to confirm that:
• —the engine terminates rather than hangs;
• —it completes without errors;
• —the answer is correct and falls within the [0, 99999] integer range.
• —Full problem verification. The final problem (after structural transformations that increase complexity) is independently verified by a dedicated solver that sequentially applies mathematical lemmas (normalisation, simplification, known identities) to reduce the graph to an expression evaluable by a symbolic engine. Mismatches with the stored answer are rejected.
• —Deduplication.
• —By graph: a SHA-256 hash of the computation graph is computed; identical graphs are detected and removed.
• —By text: problems with identical natural-language statements (SHA-256 of problem text) are removed; only the first occurrence is kept.
• —Minimum sophistication. The LLM that assigns olympiad_level also assesses problem correctness: invalid or malformed problems receive level 0, and trivial ones receive level 1. Problems with olympiad_level below 2 are excluded.
• —LLM-based suspect detection. The release pipeline runs several consistency checks against how a pool of independent LLM solvers behaved on each task. These checks flag problems where solver outcomes suggest a bug in the stored answer or statement rather than genuine difficulty — for example, tasks that no strong model can solve, tasks where multiple solvers converge on the same wrong answer, or tasks where weaker models succeed while stronger ones fail. Flagged problems are filtered out of the main release; they are not published as part of this version. The thresholds are tuned conservatively so that genuinely hard but correct problems are preserved rather than over-filtered.
• —Record size limit. Serialized records exceeding 100 KB are excluded.

Intended Use

The dataset supports multiple training and evaluation approaches:

• —Supervised fine-tuning: Training models on multi-step mathematical reasoning with verified integer answers.

• —Reinforcement learning (RLVR): Verified answers enable automatic reward signals; the llm_solvers field contains logged model answer attempts with associated verification-based scores, supporting outcome-level reinforcement learning setups.

• —Lemma selection training: The lemma_applicability annotations provide positive and negative examples for learning which lemma to apply first in a given problem. same_pattern_wrong entries serve as hard negatives — plausible-but-incorrect lemmas that match the same graph pattern as the correct one — useful for contrastive lemma-selection training.

• —Curriculum learning: Difficulty is represented along multiple independent dimensions — mathematical sophistication (olympiad_level), empirical solve rate (irt_difficulty), structural complexity (num_lemmas, num_spawns), and domain — enabling gradual difficulty progression along different axes.

• —Model trajectory shaping: Each problem's llm_solvers entry records which reference models solved it, placing the problem at a specific point in a multi-dimensional capability space defined by the set of solvers. Training a new model can be framed as steering its trajectory through this space: by selecting tasks with particular solver profiles — easy for one model and hard for another, uniquely solved by a thinking model but not a base one, consistently solved across a chosen reference set — one can guide the learning process toward specific capability regions rather than relying on a single scalar difficulty axis. For example, weighting training examples by agreement with a target profile allows shaping a model to match DeepSeek-V3.2 on number theory while preserving gpt-oss-20b-level combinatorics performance.

• —Generalization evaluation: Records carry enough structural metadata (recipe_id, seed_template_id, lemma_paths, root_lemma, olympiad_level, domain) for consumers to carve out their own structure-aware evaluation splits that test generalization to new combinations of lemmas rather than memorization of fixed solution templates.

Citation

@misc{gribov2026olympiad,
  author       = {Gribov, Mikhail},
  title        = {Olympiad Math Corpus},
  year         = {2026},
  version      = {v2.1.1},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/datasets/mihailgribov/olympiad_style_integer_math_problems}},
  license      = {CC BY 4.0}
}

License

This dataset is released under the Creative Commons Attribution 4.0 International license (CC BY 4.0).

Each record contains "license": "CC BY 4.0".