Taykhoom/cagi-variant-effect-glm-tang

GLM-Tang Task 3: CAGI Regulatory Variant Effects

This dataset packages the saturation-mutagenesis MPRA variants used for Task 3 of Tang et al. The task is zero-shot variant-effect prediction: compare a reference sequence with a matched single-nucleotide alternate sequence and test whether the model score tracks the measured regulatory effect.

Choosing a configuration

Use paper-230 for the paper-level benchmark unless reproducing a specific released model notebook that requires another sequence length. Use assay-constructs only when the physical reporter insert, rather than the historical genomic model window, is the intended sequence context.

Loading

from datasets import load_dataset

paper = load_dataset(
    "Taykhoom/cagi-variant-effect-glm-tang",
    "paper-230",
    split="test",
)

nt_context = load_dataset(
    "Taykhoom/cagi-variant-effect-glm-tang",
    "all-5994",
    split="test",
)

Official evaluation subset

The paper-230 configuration contains:

Compute Pearson correlation separately for each element. The paper summarizes HepG2 using the mean correlation across LDLR, SORT1, and F9.

The released notebooks differ in whether a model score is signed or converted to an absolute magnitude. Follow the scoring convention for the model being reproduced; the dataset does not silently modify the signed experimental label.

Sequence and allele contract

In paper-230 and the five all-* configs, sequence and alt_sequence are exact GRCh37/hg19 benchmark inputs.

• —They differ at exactly variant_index.
• —variant_index = window_length // 2 - 1.
• —source_pos is one-based.
• —source_start/source_end are zero-based, half-open sequence-window coordinates.
• —source_ref and source_alt use forward-genome hg19 orientation.

The explicit hg38 derivative is stored separately:

• —hg38_sequence
• —hg38_alt_sequence
• —chrom, start, end, pos, ref, alt, strand

No model should substitute the hg38 fields when reproducing Tang's original benchmark input.

Assay-oriented config

In assay-constructs, sequence and alt_sequence are full GEO insert constructs. F9 (343 nt), LDLR (358 nt), and SORT1 (640 nt) use forward-genomic orientation; PKLR (510 nt) uses the reverse complement. Every pair differs at exactly one focal base.

The pinned insert alignments are exact for F9, LDLR, and PKLR. SORT1 aligns across 601 genomic bases with two template/reference mismatches, at hg19 positions 109817479 and 109817590; six rows cover those loci. The builder inserts both declared alleles into the template, so none of those pairs is a no-op. Three F9 and two SORT1 variants at insert index 19 are real assay constructs just outside the exact genomic alignment; their source/assay reference is C while hg19 has G. Consequently this view has 5,061 rows, while historical_benchmark_included is true for the 5,056 rows that also pass Tang's genomic reference check.

Columns

The assay config instead uses source_file/source_row provenance, historical_source_index, historical_benchmark_included, source_reference_matches_hg19, assay_orientation, assay_template_ref, and assay_reference_matches_template. It has no full-insert genomic interval: the aligned genomic segment and vector flanks do not define one contiguous reference-genome window.

Processing decisions

The source TSV positions are one-based GRCh37/hg19 coordinates. The released notebook validates the reference base in a 4,096 nt context and retains 18,442 of 18,463 rows. This release applies that same valid-row set to every model context length.

All retained variants lift uniquely to hg38. In the all-loci configs, 1,773 MSMB rows cross a reverse-strand chain and have hg38 alleles reverse complemented. None of the 5,056 official paper rows require that transformation.

The original notebook iterates over an unsorted filesystem directory, so its historical global row order is not recoverable. This release pins Tang archive member order and stable example_id values.

Processing and verification

The input TSV checks, exact model-length extraction, one-hot reconstruction, GEO insert alignment and focal-substitution checks, liftover validation, target verification, and the root manifest contract are documented at stable repository links:

• —Processing repository: https://github.com/TaykhoomDalal/GLM-Tang-Processing/tree/main
• —Task documentation: https://github.com/TaykhoomDalal/GLM-Tang-Processing/blob/main/cagi-variant-effect/README.md
• —Reproducibility manifest: https://github.com/TaykhoomDalal/GLM-Tang-Processing/blob/main/manifest.json

Sources and citation

Tang, Z. et al. Genome Biology 26, 203 (2025). https://doi.org/10.1186/s13059-025-03674-8

Kircher, M. et al. Saturation mutagenesis of twenty disease-associated regulatory elements at single base-pair resolution. Nature Communications 10, 3583 (2019). https://doi.org/10.1038/s41467-019-11526-w

Source accession: GEO GSE126550.

License

The Tang archive is CC BY 4.0. GEO and the Kircher portal do not attach a separate Creative Commons license to all underlying assay files. Users must retain source attribution and follow the original data terms.