Datasetpublic

reasoning-degeneration-dev/gepa-exp-inputs_only-rlm-20260220-083526

gepa-exp-inputs_only-rlm-20260220-083526 GEPA prompt optimization experiment on AIME math problems. Task LM: openai/gpt-4.1-mini | Reflection LM: openai/gpt-5 | Reflection Mode: inputs_only | Last updated: 2026-02-20 19:47 UTC Results Run Method k Mode Val Score Test Acc Tokens Cost Time fixed_rlm_k3 rlm 3 inputs_only 35.56% 48.67% 645,403 $0.0000 34052s Learning Curves Experiment Config { "script_name":… See the full description on the dataset page: https://huggingface.co/datasets/reasoning-degeneration-dev/gepa-exp-inputs_only-rlm-20260220-083526.

sourceHugging Faceupdated 7mo agoView on Hugging Face

0likes39downloads

Dataset Card

gepa-exp-inputs_only-rlm-20260220-083526

GEPA prompt optimization experiment on AIME math problems.

Task LM: openai/gpt-4.1-mini | Reflection LM: openai/gpt-5 | Reflection Mode: inputs_only | Last updated: 2026-02-20 19:47 UTC

Results

Run	Method	k	Mode	Val Score	Test Acc	Tokens	Cost	Time
fixedrlmk3	rlm	3	inputs_only	35.56%	48.67%	645,403	$0.0000	34052s

Learning Curves

[image]

Experiment Config

json

{
  "script_name": "run_experiment.py",
  "model": "openai/gpt-4.1-mini",
  "reflection_lm": "openai/gpt-5",
  "reflection_mode": "inputs_only",
  "hyperparameters": {
    "task_lm": "openai/gpt-4.1-mini",
    "reflection_lm": "openai/gpt-5",
    "reflection_mode": "inputs_only",
    "seed_prompt": "You are a helpful assistant. You are given a question and you need to answer it. The answer should b..."
  },
  "input_datasets": [
    "AI-MO/aimo-validation-aime",
    "MathArena/aime_2025"
  ],
  "description": "GEPA prompt optimization experiment on AIME (reflection_mode=inputs_only)",
  "num_runs": 1,
  "last_updated": "2026-02-20 19:47 UTC"
}

Dataset Configs

This repo contains multiple configs (subsets). Load them with:

python

from datasets import load_dataset

results    = load_dataset("reasoning-degeneration-dev/gepa-exp-inputs_only-rlm-20260220-083526", "results", split="train")
traces     = load_dataset("reasoning-degeneration-dev/gepa-exp-inputs_only-rlm-20260220-083526", "traces", split="train")
val_traces = load_dataset("reasoning-degeneration-dev/gepa-exp-inputs_only-rlm-20260220-083526", "val_traces", split="train")
curves     = load_dataset("reasoning-degeneration-dev/gepa-exp-inputs_only-rlm-20260220-083526", "curves", split="train")
rlm_call_traces = load_dataset("reasoning-degeneration-dev/gepa-exp-inputs_only-rlm-20260220-083526", "rlm_call_traces", split="train")

Config	Description
`results`	Per-run summary: method, k, val/test scores, tokens, cost, optimized prompt
`traces`	Per-example test set traces with model responses and scores
`val_traces`	Per-iteration validation traces with the prompt used at each step
`curves`	Per-iteration learning curves: tokens, scores, candidate prompts
`rlm_call_traces`	Hierarchical LLM call traces: every RLM iteration, sub-call, prompt, and response

State file: `state.json`

For experiment design details, see EXPERIMENTS-DESIGN.md