raftbioworks/hierarchical_backbone_mvp_20260920_063820

Equivariant hierarchical backbone MVP

Completed 2026-09-20 using two local L40s. Six small residue-frame models completed 100k optimizer updates each; six Cartesian dense/sparse/hierarchical flow/diffusion arms completed 20k each, alongside shorter architecture, data-view and distillation experiments. W&B metrics · Full report · Reproduction guide

The central result is computational feasibility, with generation quality still limiting. A 245k-parameter equivariant frame model completed 100 full AdamW updates on 1,722,720 observed residues of symmetry-expanded PDB 6CGR at 39.90 GiB peak GPU memory. These are repeated copies of 28,712 complete deposited positions, not 1.72M unique residues or a demonstrated million-residue generator.

A separate fully gathered-edge implementation of the 16.8M-parameter FrameFlow network reaches 131,072 synthetic positions in float32 inference at 13.36 GiB, and 8,192 in checkpointed forward/backward at 13.76 GiB. It removes dense IPA, sequence attention and pair-state storage. Bounded memory does not establish linear training time or biological quality.

The IPA-mask probes retain dense storage. The fixed gate uses two ProteinMPNN sequences per backbone, ESMFold self-consistency RMSD <2 Å, pLDDT ≥70, ≥95% valid CA bonds and zero nonlocal CA clashes. This is partial computational validation; there is one training seed per arm. Confidence intervals and paired outcomes are archived. The dense teacher uses different data, capacity and training from the small from-scratch models.

Sampling matters: identical published weights pass 0/8 with linear rotation updates and 7/8 with exponential updates on matched initial noise, both without self-conditioning. The report includes corrected-schedule controls for the small models. At 512 residues the dense reference passes only 1/8 under the same limited sequence-design budget; both sparse short/long continuations pass 0/8.

A later continuation of the fully sparse 4k parent reaches 20,000 total updates. On a fresh paired 24-seed panel it passes 7/24, versus 11/24 for its 4k parent and 15/24 for the dense reference. At 512 residues, the corresponding fresh-panel results are 0/8 / 0/8 / 0/8. The new seed plan was fixed before those outputs; this is one exploratory continuation, not a second training seed.

A separate capacity-matched scalar-context pilot starts two models from the same 4k sparse parent with identical zero-initialized additions (201,600 parameters), then trains each for 2k more updates. A local MLP passes 9/24; hierarchical pooling and broadcast of the same learned scalar features passes 9/24 on fresh paired 128-residue seeds. This is short-protein evidence from one training seed, not a test of an independent coarse geometry generator.

Two longer native-backbone controls (446 and 537 residues, separate validation components) pass 2/2 with two redesigned sequences per backbone and 2/2 with their deposited sequences. The control was added after observing poor 512-residue generation; selection did not use refolding outcomes.

An inference-only packing diagnostic scales the 4k sparse parent's initial translation noise by (512/128)^(1/3). It passes 0/8 at 512 residues and is explicitly exploratory: its inference prior differs from training.

Data

The pack contains 14,096 PDB biological assemblies and 6,074,685 residue positions, including 512 selected assemblies of length 1,024–4,096. It combines Modal framediff-data/mixed816-20260919_10p and a seeded sample from framediff-large-assemblies-1024-4096. This is a protein-only subset, not the whole PDB.

data/backbones.npz stores concatenated N/CA/C/O atoms [residues,4,3] in Å, mask, chain, seq, aatype, and example offsets. Missing positions stay masked; only complete nondegenerate frames are supervised. Sequence identities support clustering, not sequence conditioning. Metadata retains source IDs and hashes.

MMseqs2 uses 30% identity and 80% bidirectional coverage. Assemblies sharing a chain cluster or PDB entry are joined before splitting: 12,597 train / 902 validation / 597 test. This is neither temporal nor a guarantee against structural homology. The frozen final test selects 26 assemblies, including ten above 1,024 residues; only two test examples are available above 2,048. See data/manifest.json and diagnostics/final_cohort_counts.json.

Files

• —data/: training pack, splits, metadata and hashes.
• —runs/: checkpoints, SHA-256 sidecars, configs, optimizer/RNG states, metrics and W&B links.
• —configs/, reproduce/: historical launch files, exploratory scripts, checks and source history.
• —source/: reusable source, dependency modules, final report and figures.
• —evaluations/: backbones, sequences, refolding, assembly diagnostics, intervals and run index.
• —diagnostics/: numerical checks, memory measurements, cohort plans and operational issues.
• —benchmarks/6cgr/: raw mmCIF, biological-assembly operators, masks and expanded arrays.
• —reference_models/: original FrameFlow weights and provenance.

Checkpoint audits verify finite model/optimizer tensors, resume fields and public LFS hashes. Early source snapshots are incomplete; exact historical replay is not claimed. The report records failures and recovered workers as well as successes.

Please credit structure authors through their PDB IDs. Licenses differ by component; see LICENSES.md. ESMFold and ProteinMPNN evaluator weights are referenced by hashes and official sources rather than republished.