mibohl/Bulk-RNA-Seq-Benchmark

Bulk RNA-seq benchmark for foundation-model evaluation: 58 tasks (25 regression, 33 classification) across 38 datasets spanning clinical outcome prediction, diagnostics, and biological/pharmacological phenotyping.

Format

One Parquet file per dataset in data/. Each row is a sample; columns are:

• —sample_id — sample accession (GSM / cell line / donor id).
• —gene columns — the native expression matrix X (gene-symbol headers).
• —y__<task> — one label column per task on that dataset. NaN = sample not in that task's cohort. Reproduce a task by selecting [gene cols] for rows where y__<task> is not NaN.
• —group__<task> — grouping key for tasks that use grouped cross-validation (donor / patient).

The number of usable rows in a task is the non-NaN count of its y__ column**, which is often smaller than the full cohort size

All expression is on a log1p scale and non-negative: mostly log1p-CPM; GTEx is log1p; CMP and GSE81538 have been converted to log1p from their native linear-TPM / log2 scales.

Loading

import pandas as pd
df = pd.read_parquet("data/CMP.parquet")
y = df["y__ic50_Bortezomib"]
mask = y.notna()
X = df.loc[mask, [c for c in df.columns if not c.startswith(("y__", "group__", "sample_id"))]]
y = y[mask]

Cross-validation splits

Most datasets have one sample per subject and can be split randomly. The following datasets contain multiple samples per patient/donor and must be split by their group__<task> column (GroupKFold / StratifiedGroupKFold), never randomly, or the same subject leaks across folds:

BeatAML, CPTAC, GSE107995, GSE156902, GSE157657, GSE193677, GSE243375, GSE278476, GSE279480, GTEx, TARGET, TCGA.

from sklearn.model_selection import GroupKFold
groups = df.loc[mask, "group__ajcc_stage_4"]          # e.g. TCGA patient barcode
for tr, te in GroupKFold(5).split(X, y, groups):
    ...

Task catalogue

Type A — Clinical & biological outcome prediction

Treatment response, recurrence, survival, drug sensitivity, staging, and molecular phenotypes.

Small (n ≤ 100)

Medium (n > 100)

Type B — Diagnostic prediction

Disease / infection presence or class predicted from the transcriptome (often a surrogate tissue such as blood or platelets).