mlfoundations-dev/a1_code_rosetta_eval_636d

mlfoundations-dev/a1coderosettaeval636d

Precomputed model outputs for evaluation.

Evaluation Results

Summary

AIME24

• —Average Accuracy: 15.67% ± 0.82%
• —Number of Runs: 10

AMC23

• —Average Accuracy: 58.50% ± 1.18%
• —Number of Runs: 10

MATH500

• —Accuracy: 74.60% | Accuracy | Questions Solved | Total Questions | |----------|-----------------|----------------| | 74.60% | 373 | 500 |

MMLUPro

• —Average Accuracy: 28.60% ± 0.00%
• —Number of Runs: 1

JEEBench

• —Average Accuracy: 42.22% ± 0.95%
• —Number of Runs: 3

GPQADiamond

• —Average Accuracy: 44.44% ± 1.65%
• —Number of Runs: 3

LiveCodeBench

• —Average Accuracy: 29.81% ± 0.94%
• —Number of Runs: 3

CodeElo

• —Average Accuracy: 5.54% ± 0.23%
• —Number of Runs: 3

CodeForces

• —Average Accuracy: 8.83% ± 0.38%
• —Number of Runs: 3