kth8/gemma-4-E2B-it-GPQA-Diamond-benchmark

Benchmark of google/gemma-4-E2B-it against GPQA-Diamond dataset. Results are averaged over 4 runs to reduce variance. Accuracy: 39.5% with Python tool. Metric Value Correct 313 Incorrect 478 Errors 1 Total samples 792 Python tool calls 209 Python tool errors 19 Total completion tokens 1,984,858