mlfoundations-dev/bespoke_stratos_eval_2e29

mlfoundations-dev/fig1allbespokestratoseval_2e29

Precomputed model outputs for evaluation.

Evaluation Results

Summary

AIME24

• —Average Accuracy: 23.67% ± 1.79%
• —Number of Runs: 10

AMC23

• —Average Accuracy: 67.75% ± 1.48%
• —Number of Runs: 10

MATH500

• —Accuracy: 78.60% | Accuracy | Questions Solved | Total Questions | |----------|-----------------|----------------| | 78.60% | 393 | 500 |

MMLUPro

• —Average Accuracy: 29.40% ± 0.00%
• —Number of Runs: 1

JEEBench

• —Average Accuracy: 44.82% ± 0.47%
• —Number of Runs: 3

GPQADiamond

• —Average Accuracy: 42.42% ± 1.33%
• —Number of Runs: 3

LiveCodeBench

• —Average Accuracy: 39.92% ± 0.41%
• —Number of Runs: 3

CodeElo

• —Average Accuracy: 13.47% ± 1.33%
• —Number of Runs: 3

CodeForces

• —Average Accuracy: 15.23% ± 1.00%
• —Number of Runs: 3

LiveCodeBenchv5

• —Average Accuracy: 30.26% ± 1.01%
• —Number of Runs: 3

AIME25

• —Average Accuracy: 21.00% ± 1.06%
• —Number of Runs: 10

HLE

• —Average Accuracy: 1.69% ± 0.14%
• —Number of Runs: 3