Datasetpublic

jasperyeoh2/pevo-msa-grch38-19way

pevo-msa-grch38-19way (dataset Hub) EN: Training data, project tables, and reproducibility artifacts for primate MSA variant-effect modeling. 中文：灵长类 MSA 变异效应建模的训练数据与项目复现材料（不含模型权重）。 Results-status note (2026-07-29). The multi-seed metrics reported below are retained as historical registry records. They use an earlier scoring protocol and cohort convention, and are not comparable to the corrected strict-v2 results used for the final project conclusions. Do not use the values… See the full description on the dataset page: https://huggingface.co/datasets/jasperyeoh2/pevo-msa-grch38-19way.

sourceHugging Facemitupdated 16d agoView on Hugging Face

0likes3.6kdownloads

Dataset Card

pevo-msa-grch38-19way (dataset Hub)

EN: Training data, project tables, and reproducibility artifacts for primate MSA variant-effect modeling. 中文： 灵长类 MSA 变异效应建模的训练数据与项目复现材料（不含模型权重）。

Results-status note (2026-07-29). The multi-seed metrics reported below are retained as historical registry records. They use an earlier scoring protocol and cohort convention, and are not comparable to the corrected strict-v2 results used for the final project conclusions. Do not use the values below for a current 10-way-versus-19-way claim; use the final project reproducibility materials with their frozen corrected-score manifests instead.


Code & experiment map	Private implementation materials are not part of this public release.
Model checkpoints	jasperyeoh2/pevo-msa-mlm-19way
Full file tree	`docs/HUGGINGFACE_DATASET_LAYOUT.md` (this repo)
Upload audit	Dataset commit history

Dataset vs model — which Hub?

text

  THIS REPO (data)                    MODEL REPO (weights)
  ────────────────                    ────────────────────
  Zarr, MAF, thesis_repro,     +      base_models/*/checkpoint-best,
  windows, conservation                configs, inventory

Quick download / 快速下载

bash

pip install -U huggingface_hub

# A) Project reproducibility materials only (~hundreds of MB)
hf download jasperyeoh2/pevo-msa-grch38-19way --repo-type dataset --local-dir data_hf \
  --include 'thesis_repro/**' 'docs/**'

# B) Training corpus (Zarr per-chromosome tar — extract after download)
hf download jasperyeoh2/pevo-msa-grch38-19way --repo-type dataset --local-dir data_hf \
  --include 'new_dataset_27feb/msaasr_dataset_10primates_full/**'

# C) Full snapshot (100GB+)
hf download jasperyeoh2/pevo-msa-grch38-19way --repo-type dataset --local-dir data_hf

Public reproducibility bundle

Download the public reproduction materials:

bash

hf download jasperyeoh2/pevo-msa-grch38-19way --repo-type dataset --local-dir data_hf \
  --include 'thesis_repro/**'
ls data_hf/thesis_repro/

Tip: The private implementation repository is intentionally not part of this public release. The existing thesis_repro/ directory name is retained only for download compatibility.

`thesis_repro/` path	Used for
`phylo_msa1_outputs/VEP_*.csv`	VEP leaderboards, strict-v2 tables (`tab:vep_`, `tab:strictv2_`)
`phylo_msa1_outputs/vep_all_base_models/*.parquet`	Per-checkpoint VEP vectors, ablation registry
`dna_foundation_benchmark_results_final/`	Feng AUROC tables/figures (`tab:feng_`, `fig:feng_`)
`thesis_manu_v2_6_artifacts/`	Matched-width strict-v2 bootstrap (May 2026)

Historical multi-seed matched-width registry (Jun 2026; earlier protocol)

Historical only. The three-arm values in this section predate the corrected target-masked, strand-averaged strict-v2 scorer. They are preserved for provenance, not as a corrected matched-width comparison.

3 seeds × 3 arms (full_19way, matched_7way, human_only) trained on AutoDL (RTX 4090), evaluated on strict-v2 cohort (n=2000).

Artifact	Path on this Hub
VEP per-variant parquets (9 runs)	`phylo_msa1/outputs/multi_seed/vep_scores/*_scores.parquet`
Bootstrap / cross-seed CSVs	`phylo_msa1/outputs/multi_seed/vep_scores/multi_seed_*.csv`
Analysis report	`phylo_msa1/outputs/multi_seed/vep_scores/multi_seed_analysis_report.md`
Training + VEP logs	`phylo_msa1/outputs/multi_seed/logs/`
Config snapshots	`phylo_msa1/outputs/multi_seed/configs/train_*.json`
Checkpoint summaries	`phylo_msa1/outputs/multi_seed/training_summaries/`
Provenance index	`phylo_msa1/outputs/multi_seed/MULTI_SEED_MANIFEST.md`

Checkpoints (~75 MB each) live on the model Hub: jasperyeoh2/pevo-msa-mlm-19way → base_models/multi_seed_{arm}_seed{seed}/checkpoint-best/

Historical cross-seed AUROC (mean ± std): full19way 0.7925 ± 0.0051 · matched7way 0.7068 ± 0.0042 · human_only 0.5085 ± 0.0020

bash

# Download scores + logs only
hf download jasperyeoh2/pevo-msa-grch38-19way --repo-type dataset --local-dir data_hf \
  --include 'phylo_msa1/outputs/multi_seed/**'

# Download one checkpoint for re-scoring
hf download jasperyeoh2/pevo-msa-mlm-19way --repo-type model --local-dir models_hf \
  --include 'base_models/multi_seed_full_19way_seed42/**'

Reproduction scripts are maintained privately and are not part of this public release.

Main data prefixes / 主要数据目录

Prefix	Contents	Size
`thesis_repro/`	Project CSV + VEP parquets + Feng summaries	small
`phylo_msa1/outputs/multi_seed/`	Multi-seed ablation VEP scores, logs, configs (Jun 2026)	~3 MB
`new_dataset_27feb/msaasr_dataset_10primates_full/bundles/`	24× `msaasr.zarr.bundle_*.tar`	~132 GB total
`new_dataset_27feb/gpnmsa_1024bp_pipeline/`	1024bp window parquets, VEP ablation (VCFs may be chunked)	mixed
`window_workspace/neutral_model_10primates/`	`merged_10primates_training.maf`	~23 GB
`conservation_pipeline/scores/`	phastCons / phyloP `.wig` per chromosome	moderate
Root `train_.parquet`, `windows_10primates_.parquet`	Multi-resolution window panels	moderate each
`msaasr.zarr/` (legacy)	Old chunk tree at repo root	prefer bundles

Project figure/table index (data side)

LaTeX label	Data on this Hub
`tab:vep_`, `tab:strictv2_`, `tab:vep_leaderboard_full`	`thesis_repro/phylo_msa1_outputs/VEP_*.csv` (+ parquets)
`fig:feng_`, `tab:feng_`	`thesis_repro/dna_foundation_benchmark_results_final/`
`fig:gnomad-tail-*`	Summary CSV is not included in this public release; score parquets are cluster-only
Retrain from scratch	`bundles/*.tar` → extract `msaasr.zarr/`

Checkpoints for scoring: model Hub base_models/full_training_*/checkpoint-best/. The private implementation mapping is not part of this public release.

MSAASR Zarr restore / 训练语料恢复

1.Download new_dataset_27feb/msaasr_dataset_10primates_full/bundles/msaasr.zarr.bundle_*.tar
2.Read README_MSAASR_BUNDLES.md in that folder
3.Extract to new_dataset_27feb/msaasr_dataset_10primates_full/msaasr.zarr/
4.Do not rely on root msaasr.zarr/ chunk trees (millions of files; slow)

Regenerate project LaTeX (minimal)

bash

cd thesis_manu_v2_6/scripts
python3 gen_strictv2_ci_artifacts.py
python3 gen_feng_combined_seq_and_l7_auroc.py
cd .. && latexmk -pdf main.tex

Citation & license

Cite this project or its associated preprint and pin this dataset commit from History. License: MIT (dataset card). gnomAD / ClinVar / EPO data remain under their original terms.