kurtos-ai/gsm-gpt2-rff

gsm-gpt2-rff

A reproducible, standardized mathematical reasoning dataset constructed from five public sources, with controllable data selection fractions (γ) using a Generalized Linear Model with Random Fourier Feature for selection.

Overview

This dataset is an aggregate of five widely used mathematical reasoning datasets:

• —deepmind/aqua_rat (raw, train split)
• —openai/gsm8k (main, train split)
• —allenai/math_qa (train split)
• —meta-math/MetaMathQA (train split)
• —microsoft/orca-math-word-problems-200k (train split)

All datasets were reformatted and standardized into a common schema with the following fields:

• —idx (int64)
• —prompt (string)
• —text (string)
• —dataset_name (string)

The repository is organized as multiple configurations, where each configuration corresponds to a different selection fraction (gamma) of the underlying candidate pool. All configurations expose a single train split.

Available Configurations

Each gamma value is a separate dataset configuration. gamma corresponds to the fraction of examples selected from the full candidate pool. Selection is based on model scores produced by the GLM + RFF, not random subsampling.

Example usage:

from datasets import load_dataset

ds = load_dataset("kurtos-ai/gsm-gpt2-rff", "gamma_0.1")
print(ds["train"][0])

{
  "idx": 0,
  "prompt": "Question: Two friends plan to walk along a 43-km trail, starting at opposite ends of the trail at the same time. If Friend P's rate is 15% faster than Friend Q's, how many kilometers will Friend P have walked when they pass each other?\nOptions: ['A)21', 'B)21.5', 'C)22', 'D)22.5', 'E)23']\nAnswer: ",
  "text": "Question: Two friends plan to walk along a 43-km trail, starting at opposite ends of the trail at the same time. If Friend P's rate is 15% faster than Friend Q's, how many kilometers will Friend P have walked when they pass each other?\nOptions: ['A)21', 'B)21.5', 'C)22', 'D)22.5', 'E)23']\nAnswer: If Q complete x kilometers, then P completes 1.15x kilometers.\nx + 1.15x = 43\n2.15x=43\nx = 43/2.15 = 20\nThen P will have have walked 1.15*20=23 km.\nThe answer is E.<|endoftext|>",
  "dataset_name": "aqua_rat"
}

Dataset Construction

Source aggregation

Each example originates from one of the five datasets listed above. The original train splits were used at fixed revisions for reproducibility. Namely,

• —deepmind/aqua_rat @ `33301c6`
• —openai/gsm8k @ `cc7b047`
• —allenai/math_qa @ `fafb9f7`
• —meta-math/MetaMathQA @ `aa4f34d`
• —microsoft/orca-math-word-problems-200k @ `29255d1`

Standardization

Each source dataset was transformed into the standardized schema (idx, prompt, text, dataset_name) using source-specific formatter configs:

• —aqua_rat
• —input_fields: ["question", "options"]
• —output_field: rationale
• —prompt_template: Question: {question}\nOptions: {options}\nAnswer:
• —gsm8k
• —input_fields: ["question"]
• —output_field: answer
• —prompt_template: Question: {question}\nAnswer:
• —math_qa
• —input_fields: ["Problem", "options"]
• —output_field: Rationale
• —prompt_template: Problem: {Problem}\nOptions: {options}\nAnswer:
• —metamathqa
• —input_fields: ["query"]
• —output_field: response
• —prompt_template: Question: {query}\nAnswer:
• —orca_math_200k
• —input_fields: ["question"]
• —output_field: answer
• —prompt_template: Question: {question}\nStep-by-step Solution:

For all sources:

• —prompt is rendered from the source-specific prompt_template
• —text is populated from the corresponding output_field
• —dataset_name records the source identifier

• —idx is an integer row id assigned after combining all standardized sources

Selection

The selection procedure was run in 4xRTX-5090.

The selection procedure was done by a Generalized Linear Model, derived from `GPT-2` embeddings with Random Fourier Features.

Number of parameters:

• —GPT-2 hidden dimension: 768
• —RFF dimension: 1024
• —Vocabulary size: 50257

The GLM predicts vocabulary logits using a linear layer over concatenated GPT-2 embeddings and RFF features. The number of trainable parameters is

• —Total: 50257 x (768 + 1024) = 90M parameters

Training of the GLM (10 epochs on the whole dataset) took:

• —2h, 24min, 24s or $8.57 at runpod's price of $3.56/h. Each selection run took approximately:
• —29 min or $1.72 at runpod's price

Dataset Statistics

Source composition by gamma configuration (train split):

The source distribution shifts substantially as gamma decreases, reflecting the scoring model's preference over datasets.

Intended Use

This dataset is intended for:

• —Training language models in mathematical reasoning tasks (grade school level).
• —Data selection research.
• —Curriculum learning experiments.
• —Scaling law studies.

Users should validate performance on held-out and external benchmarks before drawing broad generalization claims.

Associated model

We ran a GPT-2 LoRA finetuning experiment on each gamma configuration with identical compute budgets. The resulting models and their performances (loss and accuracy across several evaluation datasets) can be seen here: kurtos-ai/gsm-gpt2-rff

Limitations

• —Source datasets vary in annotation style and quality.
• —Source datasets differ largely in number of examples.
• —The selection procedure may amplify biases present in GPT-2 embeddings.
• —No test splits are included; users must evaluate on external benchmarks.

Licensing

This dataset is distributed under the Apache License 2.0.

This dataset is a derivative compilation of datasets originally released under Apache 2.0 and MIT licenses. All original licenses remain applicable to their respective portions. Users are responsible for complying with the licenses of the original datasets:

• —deepmind/aqua_rat
• —openai/gsm8k
• —allenai/math_qa
• —meta-math/MetaMathQA
• —microsoft/orca-math-word-problems-200k