Datasetpublic

reasoning-degeneration-dev/gepa-rlm-exp-failures_only-20260219-191545

gepa-rlm-exp-failures_only-20260219-191545 GEPA prompt optimization experiment on AIME math problems. Task LM: openai/gpt-4.1-mini | Reflection LM: openai/gpt-5 | Reflection Mode: failures_only | Last updated: 2026-02-19 21:50 UTC Results Run Method k Mode Val Score Test Acc Tokens Cost Time fixed_rlm_k20 rlm 20 failures_only 46.67% 30.67% 1,150,570 $0.0000 5996s Learning Curves Experiment Config { "script_name":… See the full description on the dataset page: https://huggingface.co/datasets/reasoning-degeneration-dev/gepa-rlm-exp-failures_only-20260219-191545.

sourceHugging Faceupdated 7mo agoView on Hugging Face

0likes62downloads

Dataset Card

gepa-rlm-exp-failures_only-20260219-191545

GEPA prompt optimization experiment on AIME math problems.

Task LM: openai/gpt-4.1-mini | Reflection LM: openai/gpt-5 | Reflection Mode: failures_only | Last updated: 2026-02-19 21:50 UTC

Results

Run	Method	k	Mode	Val Score	Test Acc	Tokens	Cost	Time
fixedrlmk20	rlm	20	failures_only	46.67%	30.67%	1,150,570	$0.0000	5996s

Learning Curves

[image]

Experiment Config

json

{
  "script_name": "run_experiment.py",
  "model": "openai/gpt-4.1-mini",
  "reflection_lm": "openai/gpt-5",
  "reflection_mode": "failures_only",
  "hyperparameters": {
    "task_lm": "openai/gpt-4.1-mini",
    "reflection_lm": "openai/gpt-5",
    "reflection_mode": "failures_only",
    "seed_prompt": "You are a helpful assistant. You are given a question and you need to answer it. The answer should b..."
  },
  "input_datasets": [
    "AI-MO/aimo-validation-aime",
    "MathArena/aime_2025"
  ],
  "description": "GEPA prompt optimization experiment on AIME (reflection_mode=failures_only)",
  "num_runs": 1,
  "last_updated": "2026-02-19 21:50 UTC"
}

Dataset Configs

This repo contains multiple configs (subsets). Load them with:

python

from datasets import load_dataset

results    = load_dataset("reasoning-degeneration-dev/gepa-rlm-exp-failures_only-20260219-191545", "results", split="train")
traces     = load_dataset("reasoning-degeneration-dev/gepa-rlm-exp-failures_only-20260219-191545", "traces", split="train")
val_traces = load_dataset("reasoning-degeneration-dev/gepa-rlm-exp-failures_only-20260219-191545", "val_traces", split="train")
curves     = load_dataset("reasoning-degeneration-dev/gepa-rlm-exp-failures_only-20260219-191545", "curves", split="train")

Config	Description
`results`	Per-run summary: method, k, val/test scores, tokens, cost, optimized prompt
`traces`	Per-example test set traces with model responses and scores
`val_traces`	Per-iteration validation traces with the prompt used at each step
`curves`	Per-iteration learning curves: tokens, scores, candidate prompts

State file: `state.json`

For experiment design details, see EXPERIMENTS-DESIGN.md