mlfoundations-dev/bespoke_stratos_3k_eval_636d

mlfoundations-dev/fig1scalingbespokestratos3keval636d

Precomputed model outputs for evaluation.

Evaluation Results

Summary

AIME24

• —Average Accuracy: 17.67% ± 1.64%
• —Number of Runs: 10

AMC23

• —Average Accuracy: 60.50% ± 1.76%
• —Number of Runs: 10

MATH500

• —Accuracy: 77.60% | Accuracy | Questions Solved | Total Questions | |----------|-----------------|----------------| | 77.60% | 388 | 500 |

MMLUPro

• —Average Accuracy: 28.20% ± 0.00%
• —Number of Runs: 1

JEEBench

• —Average Accuracy: 41.12% ± 0.67%
• —Number of Runs: 3

GPQADiamond

• —Average Accuracy: 43.27% ± 0.50%
• —Number of Runs: 3

LiveCodeBench

• —Average Accuracy: 30.72% ± 0.39%
• —Number of Runs: 3

CodeElo

• —Average Accuracy: 5.54% ± 0.45%
• —Number of Runs: 3

CodeForces

• —Average Accuracy: 7.65% ± 0.07%
• —Number of Runs: 3