algorithmic-reasoning
gepa-rlm-exp-algorithmic-20260219-191545
gepa-rlm-exp-algorithmic-20260219-191545
GEPA prompt optimization experiment on AIME math problems.
Task LM: openai/gpt-4.1-mini | Reflection LM: openai/gpt-5 | Reflection Mode: algorithmic | Last updated: 2026-02-19 21:34 UTC
Results
Run
Method
k
Mode
Val Score
Test Acc
Tokens
Cost
Time
fixed_rlm_k20
rlm
20
algorithmic
48.89%
26.67%
932,391
$0.0000
5209s
Learning Curves
Experiment Config
{
"script_name": "run_experiment.py"… See the full description on the dataset page: https://huggingface.co/datasets/reasoning-degeneration-dev/gepa-rlm-exp-algorithmic-20260219-191545.gepa-exp-algorithmic-vanilla-20260220-083526
gepa-exp-algorithmic-vanilla-20260220-083526
GEPA prompt optimization experiment on AIME math problems.
Task LM: openai/gpt-4.1-mini | Reflection LM: openai/gpt-5 | Reflection Mode: algorithmic | Last updated: 2026-02-20 21:20 UTC
Results
Run
Method
k
Mode
Val Score
Test Acc
Tokens
Cost
Time
fixed_vanilla_k3
vanilla
3
algorithmic
35.56%
46.67%
77,464
$0.2182
3126s
Learning Curves
Experiment Config
{
"script_name":… See the full description on the dataset page: https://huggingface.co/datasets/reasoning-degeneration-dev/gepa-exp-algorithmic-vanilla-20260220-083526.algorithmic-sft-training-data-v1
algorithmic-sft-training-data-v1
Algorithmic SFT training data: deterministic step-by-step traces for 5 domains (countdown, formal_logic, long_arithmetic, conlang_morphology, cellular_automata) across multiple algorithm variants. Programmatically generated — no LLM involved.
Dataset Info
Rows: 63000
Columns: 8
Columns
Column
Type
Description
question
Value('string')
The problem statement presented to the model
answer
Value('string')
The correct… See the full description on the dataset page: https://huggingface.co/datasets/reasoning-degeneration-dev/algorithmic-sft-training-data-v1.gepa-exp-algorithmic-rlm-20260220-083526
gepa-exp-algorithmic-rlm-20260220-083526
GEPA prompt optimization experiment on AIME math problems.
Task LM: openai/gpt-4.1-mini | Reflection LM: openai/gpt-5 | Reflection Mode: algorithmic | Last updated: 2026-02-20 21:25 UTC
Results
Run
Method
k
Mode
Val Score
Test Acc
Tokens
Cost
Time
fixed_rlm_k3
rlm
3
algorithmic
37.78%
44.00%
400,201
$0.0000
3483s
Learning Curves
Experiment Config
{
"script_name": "run_experiment.py"… See the full description on the dataset page: https://huggingface.co/datasets/reasoning-degeneration-dev/gepa-exp-algorithmic-rlm-20260220-083526.algorithmic-reasoning-seed
Dataset Card for Algorithmic Reasoning (seed)
Note: This dataset is WIP and most question's answer section is empty or incomplete! See also "Other Known Limitations" section
Warning: If you somehow do use this dataset, remember to NOT do any eval after training on the questions in this dataset!
Dataset Summary
Dataset to help LLM learn how to reason about code, especially on algorithmic tasks, by seeing human demostration.
Supported Tasks and Leaderboards
[More… See the full description on the dataset page: https://huggingface.co/datasets/lemonteaa/algorithmic-reasoning-seed.algorithmic-sft-training-configs-v1
algorithmic-sft-training-configs-v1
LlamaFactory training configs. All cutoff_len=32768. Countdown configs use new equation-answer format.
Dataset Info
Rows: 17
Columns: 7
Columns
Column
Type
Description
config_name
Value('string')
YAML filename
domain
Value('string')
No description provided
is_distillation
Value('bool')
No description provided
yaml_content
Value('string')
Full YAML config
model_name
Value('string')
No description provided… See the full description on the dataset page: https://huggingface.co/datasets/reasoning-degeneration-dev/algorithmic-sft-training-configs-v1.