darlednik/geneb-tasks

GENEB — Genomic Embedding Benchmark (task data)

Task-level sequence classification data for GENEB, a multi-task benchmark for DNA sequence encoders introduced in the paper: [GENEB: Why Genomic Models Are Hard to Compare](https://huggingface.co/papers/2606.04525).

• —Paper: https://huggingface.co/papers/2606.04525
• —Source code: GitHub - darlednik/GENEB
• —Leaderboard: Hugging Face Space

GENEB evaluates frozen representations from 40 genomic foundation models across 100 tasks in 13 functional categories using a unified probing-based protocol. This repository release contains train and test partitions only.

Sample Usage

To sync the GENEB task data locally using the reference tools:

python3 -m pip install "huggingface_hub>=0.24"
python3 tools/sync_geneb_dataset.py download --local_dir ./GENEB_data

Record schema

Each example is a single row with:

• —`text` — nucleotide sequence (DNA over {A,C,G,T}; length and windowing are defined by the source task).
• —`label` — discrete class label for the assay or prediction target.

Train and test examples are provided in separate files per task (train.csv, test.csv). Auxiliary splits present in some upstream releases (e.g. dev, validation) are excluded from this distribution and are not used in the GENEB evaluation protocol.

Evaluation protocol (summary)

GENEB scores models on these tasks under a shared protocol (full specification in the benchmark repository):

Participants should use the released train/test assignments as-is; relabeling, resplitting, or training on the test partition is outside the protocol.

Task categories

Repository layout and Dataset Viewer

Data are organized as tasks/<task_id>/train.csv and tasks/<task_id>/test.csv with columns text and label.

On the Hub Dataset Viewer, select:

1. 1.Subset (config_name) — one benchmark task (short slug, e.g. NT_H3; see index below).
2. 2.Split — train or test.

Task index

Citation

@misc{ledneva2026genebgenomicmodelshard,
  title         = {GENEB: Why Genomic Models Are Hard to Compare},
  author        = {Daria Ledneva and Mikhail Nuridinov and Denis Kuznetsov},
  year          = {2026},
  eprint        = {2606.04525},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2606.04525}
}

Contact

GENEB is led and maintained by Daria Ledneva. For questions, suggestions, or model submissions, please contact a.ledn2026@gmail.com.