archiitecture/initial-dynamic-proteins

Initial 2,000-protein dataset

This is the canonical local root for the first complete router dataset: 1,000 single-dominant structured-state proteins (label 0) and 1,000 dynamic or heterogeneous-state proteins (label 1). The fixed split is 1,400 train, 300 validation, and 300 test proteins.

Place Colab's completed ESMFold result files (<sequence_sha256>.npz) in esmfold_results/. Then import them with:

uv run python scripts/esmfold_dataset.py import

The importer validates every NPZ against esmfold_input_manifest.csv and writes the reusable package-native bundles into embeddings/esmfold_v1/. It also writes embeddings/esmfold_v1/manifest.parquet, the index consumed by the training commands. Do not train directly from esmfold_results/; it is only a transfer/staging location and can be deleted after a successful import.

Key files:

• —catalog.parquet and catalog.csv: canonical labels, sequences, metadata, and fixed split assignments.
• —catalog_esmfold_eligible.parquet and .csv: the 1,988 rows with verified ESMFold bundles; use these for embedding-only or combined model training.
• —esmfold_input_manifest.csv: exact sequence hashes and model input contract.
• —embeddings/esmfold_v1/manifest.parquet: verified embedding-bundle index.
• —embeddings/esmfold_v1/excluded.csv: the 12 sequences outside ESMFold's 1,022-residue limit.

For CPU/MPS training, pass --dataset data/initial_2000_dataset/catalog.parquet and --bundle-manifest data/initial_2000_dataset/embeddings/esmfold_v1/manifest.parquet.