Datasetpublic

reasoning-degeneration-dev/gepa-rlm-exp-domain_heuristics-20260219-191545

gepa-rlm-exp-domain_heuristics-20260219-191545 GEPA prompt optimization experiment on AIME math problems. Task LM: openai/gpt-4.1-mini | Reflection LM: openai/gpt-5 | Reflection Mode: domain_heuristics | Last updated: 2026-02-19 21:44 UTC Results Run Method k Mode Val Score Test Acc Tokens Cost Time fixed_rlm_k20 rlm 20 domain_heuristics 55.56% 28.00% 1,168,868 $0.0000 6304s Learning Curves Experiment Config {… See the full description on the dataset page: https://huggingface.co/datasets/reasoning-degeneration-dev/gepa-rlm-exp-domain_heuristics-20260219-191545.

sourceHugging Faceupdated 7mo agoView on Hugging Face

0likes58downloads

Dataset Card

gepa-rlm-exp-domain_heuristics-20260219-191545

GEPA prompt optimization experiment on AIME math problems.

Task LM: openai/gpt-4.1-mini | Reflection LM: openai/gpt-5 | Reflection Mode: domain_heuristics | Last updated: 2026-02-19 21:44 UTC

Results

Run	Method	k	Mode	Val Score	Test Acc	Tokens	Cost	Time
fixedrlmk20	rlm	20	domain_heuristics	55.56%	28.00%	1,168,868	$0.0000	6304s

Learning Curves

[image]

Experiment Config

json

{
  "script_name": "run_experiment.py",
  "model": "openai/gpt-4.1-mini",
  "reflection_lm": "openai/gpt-5",
  "reflection_mode": "domain_heuristics",
  "hyperparameters": {
    "task_lm": "openai/gpt-4.1-mini",
    "reflection_lm": "openai/gpt-5",
    "reflection_mode": "domain_heuristics",
    "seed_prompt": "You are a helpful assistant. You are given a question and you need to answer it. The answer should b..."
  },
  "input_datasets": [
    "AI-MO/aimo-validation-aime",
    "MathArena/aime_2025"
  ],
  "description": "GEPA prompt optimization experiment on AIME (reflection_mode=domain_heuristics)",
  "num_runs": 1,
  "last_updated": "2026-02-19 21:44 UTC"
}

Dataset Configs

This repo contains multiple configs (subsets). Load them with:

python

from datasets import load_dataset

results    = load_dataset("reasoning-degeneration-dev/gepa-rlm-exp-domain_heuristics-20260219-191545", "results", split="train")
traces     = load_dataset("reasoning-degeneration-dev/gepa-rlm-exp-domain_heuristics-20260219-191545", "traces", split="train")
val_traces = load_dataset("reasoning-degeneration-dev/gepa-rlm-exp-domain_heuristics-20260219-191545", "val_traces", split="train")
curves     = load_dataset("reasoning-degeneration-dev/gepa-rlm-exp-domain_heuristics-20260219-191545", "curves", split="train")

Config	Description
`results`	Per-run summary: method, k, val/test scores, tokens, cost, optimized prompt
`traces`	Per-example test set traces with model responses and scores
`val_traces`	Per-iteration validation traces with the prompt used at each step
`curves`	Per-iteration learning curves: tokens, scores, candidate prompts

State file: `state.json`

For experiment design details, see EXPERIMENTS-DESIGN.md