mlfoundations-dev/a1_code_coder_stat_eval_636d
mlfoundations-dev/a1_code_coder_stat_eval_636d Precomputed model outputs for evaluation. Evaluation Results Summary Metric AIME24 AMC23 MATH500 MMLUPro JEEBench GPQADiamond LiveCodeBench CodeElo CodeForces Accuracy 14.3 59.2 75.6 32.0 40.1 42.3 21.5 10.5 10.2 AIME24 Average Accuracy: 14.33% ± 1.25% Number of Runs: 10 Run Accuracy Questions Solved Total Questions 1 10.00% 3 30 2 16.67% 5 30 3 20.00% 6… See the full description on the dataset page: https://huggingface.co/datasets/mlfoundations-dev/a1_code_coder_stat_eval_636d.
010
mlfoundations-dev/a1codecoderstateval_636d
Precomputed model outputs for evaluation.
Evaluation Results
Summary
AIME24
- Average Accuracy: 14.33% ± 1.25%
- Number of Runs: 10
AMC23
- Average Accuracy: 59.25% ± 1.37%
- Number of Runs: 10
MATH500
- Accuracy: 75.60% | Accuracy | Questions Solved | Total Questions | |----------|-----------------|----------------| | 75.60% | 378 | 500 |
MMLUPro
- Average Accuracy: 32.00% ± 0.00%
- Number of Runs: 1
JEEBench
- Average Accuracy: 40.15% ± 0.73%
- Number of Runs: 3
GPQADiamond
- Average Accuracy: 42.26% ± 1.13%
- Number of Runs: 3
LiveCodeBench
- Average Accuracy: 21.46% ± 1.20%
- Number of Runs: 3
CodeElo
- Average Accuracy: 10.49% ± 1.29%
- Number of Runs: 3
CodeForces
- Average Accuracy: 10.23% ± 1.18%
- Number of Runs: 3