hmnshudhmn24/llm-benchmarks-capabilities-2020-2026

📊 LLM Benchmarks & Capabilities 2020–2026

The most comprehensive open dataset tracking the evolution of Large Language Models — from GPT-3 to GPT-5.5, Claude Opus 4.7, Gemini 3.5, and beyond.

🧭 Overview

This dataset captures the complete LLM landscape from 2020 to 2026 across five dimensions:

• —🤖 113 models from 25+ organizations
• —📈 17 benchmarks tracking capability growth over time
• —💰 Monthly API pricing showing 100x+ cost reductions
• —⚙️ Training compute estimates validating scaling laws
• —🏁 57 capability milestones marking key inflection points

Designed for ML researchers, AI practitioners, policy analysts, and data scientists working on LLM-related problems — trend analysis, capability forecasting, cost-performance modeling, and competitive intelligence.

📁 Dataset Files

Total: ~2,750 rows · ~250 KB

🏢 Organizations Covered

Closed Frontier OpenAI · Anthropic · Google DeepMind · xAI · Microsoft

Open Weights Meta · DeepSeek · Mistral · Alibaba Qwen · 01.AI · TII

Early Era Chinchilla · PaLM · BLOOM · OPT · GLaM · Switch Transformer

Chinese Frontier WuDao 2.0 · ERNIE 3.0 · HyperCLOVA · YaLM 100B

📐 Benchmarks Tracked

⚡ Quick Start

import pandas as pd

# Load models catalog
models = pd.read_csv("hf://datasets/hmnshudhmn24/llm-benchmarks-capabilities-2020-2026/models_catalog.csv")

# Load benchmark scores
scores = pd.read_csv("hf://datasets/hmnshudhmn24/llm-benchmarks-capabilities-2020-2026/benchmark_scores.csv")

# Load pricing history
pricing = pd.read_csv("hf://datasets/hmnshudhmn24/llm-benchmarks-capabilities-2020-2026/pricing_history.csv")

# Load compute estimates
compute = pd.read_csv("hf://datasets/hmnshudhmn24/llm-benchmarks-capabilities-2020-2026/compute_estimates.csv")

# Load capability milestones
milestones = pd.read_csv("hf://datasets/hmnshudhmn24/llm-benchmarks-capabilities-2020-2026/capability_milestones.csv")

print(models.head())

💡 Suggested Use Cases

1. 1.Benchmark Saturation Forecasting — MMLU went from 32% (2020) to 95% (2025). Predict saturation for GPQA Diamond and SWE-Bench.
2. 2.Price vs Capability Analysis — Plot Arena ELO vs blended API price. Track Pareto-optimal models per quarter.
3. 3.Open vs Closed Model Gap — Measure the capability gap between open weights and closed models across time.
4. 4.Scaling Law Validation — Plot training FLOPs vs benchmark scores. Test empirical scaling exponents.
5. 5.Reasoning Model Premium — Measure score lift of reasoning models vs chat models on math and coding benchmarks.
6. 6.Cost Reduction Trajectory — Track price per Arena ELO point over time. How fast is intelligence-per-dollar growing?
7. 7.Competitive Organization Analysis — Head-to-head benchmark matrix per organization per quarter.
8. 8.Chinese vs Western Capability Race — DeepSeek, Qwen, ERNIE vs OpenAI, Anthropic, Google.