datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
SuperGPQAThis repository contains the data presented in SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplines.
Tutorials for submitting to the official leadboard
coming soon
📜 License
SuperGPQA is a composite dataset that includes both original content and portions of data derived from other sources. The dataset is made available under the Open Data Commons Attribution License (ODC-BY), which asserts no copyright over the underlying content.
This means that while the… See the full description on the dataset page: https://huggingface.co/datasets/m-a-p/SuperGPQA.SuperGPQA-RecordsSuperGPQAdeepseek-reasoner-full-supergpqa-r1TESSY-SuperGPQA-3K
TESSY-SuperGPQA-3K
📄 Paper Link
|
🔗 GitHub Repository
📣 Paper
🎉 Accepted at ICML 2026!
How to Fine-Tune a Reasoning Model? A Teacher–Student Cooperation Framework to Synthesize Student-Consistent SFT Data
🚀 Overview
We construct a programming contest training dataset for Qwen3-8B by leveraging GPT-OSS-120B as the teacher model. The synthesized data preserves the strong reasoning capabilities of GPT-OSS-120B, while being aligned with the… See the full description on the dataset page: https://huggingface.co/datasets/CoopReason/TESSY-SuperGPQA-3K.super_gpqa_mnlp_m3atlas-29-supergpqa-records-as-a-candidate-pool
29. Is SuperGPQA-Records a candidate pool an orchestrator can learn from?
1. Question and links
Does a candidate pool built from SuperGPQA-Records (128 hard questions, eight candidates each from the eight strongest single-shot models of the records) leave room for an untrained orchestrator, Qwen3.6-27B or Qwen3.5-9B, to learn: how does each model's greedy accuracy move from no candidate to eight against the pool's coverage? Report source:… See the full description on the dataset page: https://huggingface.co/datasets/t2ance/atlas-29-supergpqa-records-as-a-candidate-pool.ReasoningIntensiveLoose_with_SuperGPQAsupergpqa_10k_seed1_claude_gemini_gpt_metricssystem_prompts_SuperGPQA-26000xSFT system prompts dataset generated using openai/gpt-oss-120b and m-a-p/SuperGPQA dataset.
Each instance follows this format:
{
"uuid": "000192f411a04f13858d69834a44ae01",
"messages": [
{
"role": "system",
"content": "You are a system prompt generator."},
{
"role": "user",
"content": "Write a system prompt that defines an AI researcher who is a leading authority in Science, specifically in Physics and Quantum Mechanics."
},
{
"role":… See the full description on the dataset page: https://huggingface.co/datasets/kth8/system_prompts_SuperGPQA-26000x.gemma-4-E2B-it-SuperGPQA-benchmarkBenchmark of google/gemma-4-E2B-it against SuperGPQA dataset. None
Accuracy: 32.7% with Python tool.
Metric
Value
Correct
328
Incorrect
666
Errors
8
Total samples
1002
Python tool calls
274
Python tool errors
22
Total completion tokens
1,999,635
SuperGPQA_PhysQwen3.5-4B-Claude-Opus-Reasoning-Distill-SuperGPQA-benchmarkBenchmark of TeichAI/Qwen3.5-4B-Claude-Opus-Reasoning-Distill against m-a-p/SuperGPQA dataset.
Accuracy: 41.5% with Python tool.
Metric
Value
Correct
415
Incorrect
573
Errors
11
Total samples
999
Python tool calls
2527
Total completion tokens
4,149,159
Raw stats:
{
"accuracy": 0.415,
"correct": 415,
"incorrect": 573,
"error": 11,
"total": 999,
"python_tool_calls": 2527,
"completion_tokens": 4149159
}
SuperGPQAqwen-instruct-synthetic_1_stem_only-sft-full-supergpqa-r1llama3.1-instruct-synthetic_1_stem_only-full-supergpqa-r1gpt-oss-20b-supergpqa-benchmarkBenchmark of openai/gpt-oss-20b against m-a-p/SuperGPQA dataset.
Accuracy: 44.9% with Python tool.
Metric
Value
Correct
898
Incorrect
1098
Errors
4
Total samples
2000
Python tool calls
2084
Python tool errors
117
Total completion tokens
3,829,112
Raw stats:
{
"accuracy": 0.449,
"correct": 898,
"incorrect": 1098,
"error": 4,
"total": 2000,
"python_tool_calls": 2084,
"python_tool_errors": 117,
"completion_tokens": 3829112
}
gpt-oss-120b-SuperGPQA-benchmarkBenchmark of openai/gpt-oss-120b against m-a-p/SuperGPQA dataset.
Accuracy: 54.900000000000006% with Python tool.
Metric
Value
Correct
549
Incorrect
448
Errors
3
Total samples
1000
Python tool calls
889
Total completion tokens
1,301,333
Raw stats:
{
"accuracy": 0.549,
"correct": 549,
"incorrect": 448,
"error": 3,
"total": 1000,
"python_tool_calls": 889,
"completion_tokens": 1301333
}
supergpqa_2k-classifiedQwen3.5-9B-SuperGPQA-benchmarkBenchmark of Qwen/Qwen3.5-9B against m-a-p/SuperGPQA dataset.
Accuracy: 52.7% with Python tool.
Metric
Value
Correct
527
Incorrect
469
Errors
4
Total samples
1000
Python tool calls
1317
Total completion tokens
2,702,049
Raw stats:
{
"accuracy": 0.527,
"correct": 527,
"incorrect": 469,
"error": 4,
"total": 1000,
"python_tool_calls": 1317,
"completion_tokens": 2702049
}
SuperGPQAsupergpqa_non_calc_middle_val_1kSuperGPQAgemma-4-E4B-it-SuperGPQA-benchmarkBenchmark of google/gemma-4-E4B-it against m-a-p/SuperGPQA dataset.
Accuracy: 38.1% with Python tool.
Metric
Value
Correct
761
Incorrect
1239
Errors
0
Total samples
2000
Python tool calls
200
Python tool errors
10
Total completion tokens
4,253,773
Raw stats:
{
"accuracy": 0.381,
"correct": 761,
"incorrect": 1239,
"error": 0,
"total": 2000,
"python_tool_calls": 200,
"python_tool_errors": 10,
"completion_tokens": 4253773
}
Qwen3.6-27B-AWQ-BF16-INT4-SuperGPQA-benchmarkBenchmark of cyankiwi/Qwen3.6-27B-AWQ-BF16-INT4 against m-a-p/SuperGPQA dataset.
Accuracy: 69.2% with Python tool.
Metric
Value
Correct
692
Incorrect
295
Errors
13
Total samples
1000
Python tool calls
1508
Total completion tokens
3,806,045
Raw stats:
{
"accuracy": 0.692,
"correct": 692,
"incorrect": 295,
"error": 13,
"total": 1000,
"python_tool_calls": 1508,
"completion_tokens": 3806045
}
llama3.1-instruct-synthetic_1_math_only-full-supergpqa-r1SuperGPQA-SmolLM3-3Bsupergpqa_10k_seed1_Qwen_gemma_granite_FP8_metricsSuperGPQAThis repository contains the data presented in SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplines.
Tutorials for submitting to the official leadboard
coming soon
📜 License
SuperGPQA is a composite dataset that includes both original content and portions of data derived from other sources. The dataset is made available under the Open Data Commons Attribution License (ODC-BY), which asserts no copyright over the underlying content.
This means that while the… See the full description on the dataset page: https://huggingface.co/datasets/Medusan/SuperGPQA.llama3.1-instruct-synthetic_1-sft-full-supergpqa-r1