CoolFace
Datasetpublic

JudyLi1122/benchpress-score-matrix

BenchPress Score Matrix This dataset contains the public model-by-benchmark score matrix used by BenchPress. The release is a tabular artifact: model metadata, benchmark metadata, one row per observed score, and the paper-canonical dense subset used in the BenchPress experiments. The source repository is microsoft/benchpress. This export was generated from commit 5be3b4eddf0188721ff25f00713b589b2cbed8e0. Files File Contents data/scores_all.csv /… See the full description on the dataset page: https://huggingface.co/datasets/JudyLi1122/benchpress-score-matrix.

sourceHugging Facecdla-permissive-2.0updated 3mo agoView on Hugging Face
0likes16downloads
Dataset Card

BenchPress Score Matrix

This dataset contains the public model-by-benchmark score matrix used by BenchPress. The release is a tabular artifact: model metadata, benchmark metadata, one row per observed score, and the paper-canonical dense subset used in the BenchPress experiments.

The source repository is `microsoft/benchpress`. This export was generated from commit 5be3b4eddf0188721ff25f00713b589b2cbed8e0.

Files

FileContents
data/scores_all.csv / .parquetAll numeric score rows in the audit pool, with source and audit metadata.
data/scores_paper.csv / .parquetLong-form rows for the paper-canonical matrix.
data/models.csv / .parquetModel metadata and canonical evaluation settings.
data/benchmarks.csv / .parquetBenchmark metadata and canonical benchmark settings.
data/score_matrix_paper_wide.csvWide model × benchmark matrix for the paper-canonical subset.
metadata.jsonExport counts, source commit, and matrix construction metadata.

Quick start

python
from datasets import load_dataset

scores = load_dataset("microsoft/benchpress-score-matrix", "scores_paper")["train"].to_pandas()
models = load_dataset("microsoft/benchpress-score-matrix", "models")["train"].to_pandas()
benchmarks = load_dataset("microsoft/benchpress-score-matrix", "benchmarks")["train"].to_pandas()

For a complete audit-pool table:

python
scores_all = load_dataset("microsoft/benchpress-score-matrix", "scores_all")["train"].to_pandas()

Schema

scores_all and scores_paper include:

  • —model_id, model_name, provider
  • —benchmark_id, benchmark_name, category, metric
  • —score
  • —reference_url, source_type, audit_status, matches_canonical
  • —reported_setting_json, notes

models and benchmarks include an in_paper_matrix flag that identifies rows retained by the paper-canonical threshold filter.

Matrix construction

The paper-canonical matrix applies the BenchPress construction pipeline: audit-status filtering, canonical representative selection, and the iterative threshold filter. Current export counts:

  • —audit pool: 189 models, 316 benchmarks, 4903 numeric score rows
  • —paper matrix: 84 models × 133 benchmarks, 2604 observed cells (23.3% fill)

Caveats

Scores come from heterogeneous public sources: model cards, official blogs, technical reports, benchmark leaderboards, and third-party aggregators. Each row keeps the source URL, source type, audit status, and canonical-setting match flag so downstream users can choose their own filtering policy.