openeurollm/prelude-evaluations

OpenEuroLLM Leaderboard

Interactive plots of normalized average downstream performance vs. tokens trained, across checkpoints of the OpenEuroLLM prelude/datamix runs and comparison models (Apertus, Olmo3, Qwen, Llama).

Scores are min-max normalized per (benchmark, metric) pair, then averaged across languages within each benchmark and across benchmarks. Tabs:

• —Aggregate — overall normalized performance vs. tokens trained
• —By Language — one small chart per language
• —By Task — one small chart per benchmark
• —Raw results — un-normalized per-(benchmark, language) scores for a selected model

Run locally

pip install -r requirements.txt
python main.py

Files

• —main.py — the Gradio app
• —leaderboard_data.json — precomputed normalized scores per (data, iter, tokens_B)
• —language_data.json — precomputed normalized scores per (data, iter, tokens_B, language)
• —task_data.json — precomputed normalized scores per (data, iter, tokens_B, benchmark)
• —raw_data.json — raw (un-normalized) scores per (data, iter, tokens_B, benchmark, language)
• —requirements.txt — pinned dependencies for the HF Space