Datasetpublic

reasoning-degeneration-dev/gepa-exp-contrastive-vanilla-20260220-083526

gepa-exp-contrastive-vanilla-20260220-083526 GEPA prompt optimization experiment on AIME math problems. Task LM: openai/gpt-4.1-mini | Reflection LM: openai/gpt-5 | Reflection Mode: contrastive | Last updated: 2026-02-20 18:21 UTC Results Run Method k Mode Val Score Test Acc Tokens Cost Time fixed_vanilla_k3 vanilla 3 contrastive 31.11% 44.00% 51,600 $0.1236 32317s Learning Curves Experiment Config { "script_name":… See the full description on the dataset page: https://huggingface.co/datasets/reasoning-degeneration-dev/gepa-exp-contrastive-vanilla-20260220-083526.

sourceHugging Faceupdated 7mo agoView on Hugging Face

0likes52downloads

Dataset Card

gepa-exp-contrastive-vanilla-20260220-083526

GEPA prompt optimization experiment on AIME math problems.

Task LM: openai/gpt-4.1-mini | Reflection LM: openai/gpt-5 | Reflection Mode: contrastive | Last updated: 2026-02-20 18:21 UTC

Results

Run	Method	k	Mode	Val Score	Test Acc	Tokens	Cost	Time
fixedvanillak3	vanilla	3	contrastive	31.11%	44.00%	51,600	$0.1236	32317s

Learning Curves

[image]

Experiment Config

json

{
  "script_name": "run_experiment.py",
  "model": "openai/gpt-4.1-mini",
  "reflection_lm": "openai/gpt-5",
  "reflection_mode": "contrastive",
  "hyperparameters": {
    "task_lm": "openai/gpt-4.1-mini",
    "reflection_lm": "openai/gpt-5",
    "reflection_mode": "contrastive",
    "seed_prompt": "You are a helpful assistant. You are given a question and you need to answer it. The answer should b..."
  },
  "input_datasets": [
    "AI-MO/aimo-validation-aime",
    "MathArena/aime_2025"
  ],
  "description": "GEPA prompt optimization experiment on AIME (reflection_mode=contrastive)",
  "num_runs": 1,
  "last_updated": "2026-02-20 18:21 UTC"
}

Dataset Configs

This repo contains multiple configs (subsets). Load them with:

python

from datasets import load_dataset

results    = load_dataset("reasoning-degeneration-dev/gepa-exp-contrastive-vanilla-20260220-083526", "results", split="train")
traces     = load_dataset("reasoning-degeneration-dev/gepa-exp-contrastive-vanilla-20260220-083526", "traces", split="train")
val_traces = load_dataset("reasoning-degeneration-dev/gepa-exp-contrastive-vanilla-20260220-083526", "val_traces", split="train")
curves     = load_dataset("reasoning-degeneration-dev/gepa-exp-contrastive-vanilla-20260220-083526", "curves", split="train")
rlm_call_traces = load_dataset("reasoning-degeneration-dev/gepa-exp-contrastive-vanilla-20260220-083526", "rlm_call_traces", split="train")

Config	Description
`results`	Per-run summary: method, k, val/test scores, tokens, cost, optimized prompt
`traces`	Per-example test set traces with model responses and scores
`val_traces`	Per-iteration validation traces with the prompt used at each step
`curves`	Per-iteration learning curves: tokens, scores, candidate prompts
`rlm_call_traces`	Hierarchical LLM call traces: every RLM iteration, sub-call, prompt, and response

State file: `state.json`

For experiment design details, see EXPERIMENTS-DESIGN.md