hugging-science/jain-clinical-antibody-polyreactivity

Jain Clinical Antibody Polyreactivity Dataset (Novo Nordisk Parity Benchmark)

Dataset Description

• —Homepage: Hugging Science Organization
• —Repository (this implementation): The-Obstacle-Is-The-Way/antibody_training_pipeline_ESM
• —Upstream: ludocomito/antibody_training_pipeline_ESM
• —Paper (Original Dataset): Jain et al. 2017, PNAS
• —Paper (Preprocessing Methodology): Sakhnini et al. 2025, bioRxiv
• —Point of Contact: Hugging Science

Dataset Summary

This dataset contains 86 clinical-stage antibody heavy chain variable domain (VH) sequences with binary polyreactivity labels, preprocessed to reproduce the benchmark results from Sakhnini et al. 2025 (Novo Nordisk & University of Cambridge). The original dataset was published by Jain et al. 2017 and contains biophysical measurements for 137 FDA-approved or late-stage clinical antibodies.

This is a benchmark test set for evaluating the ESM-1v + Logistic Regression model trained on the Boughter dataset. We achieve EXACT PARITY with Novo's published results: confusion matrix `[[40, 17], [10, 19]]`, 68.60% accuracy.

Key Features

• —Organism: Human (clinical-stage antibodies from approved drugs and clinical trials)
• —Molecule Type: Antibody heavy chain variable domain (VH)
• —Assay: ELISA polyreactivity panel (6 ligands: cardiolipin, KLH, LPS, ssDNA, dsDNA, insulin)
• —Labels: Binary classification (0 = specific, 1 = non-specific/polyreactive)
• —Benchmark: Novo Nordisk Figure S14A (68.60% accuracy)
• —Size: 86 antibodies (subset from 137 total)
• —Balance: 57 specific (66.3%) / 29 non-specific (33.7%)

Supported Tasks and Leaderboards

• —Binary Classification: Predicting antibody polyreactivity from VH sequence
• —Benchmark: Novo Nordisk parity benchmark (68.60% accuracy, confusion matrix [[40, 17], [10, 19]])

Languages

Protein sequences (amino acid alphabet)

⚠️ Important: Reverse-Engineered Preprocessing

CRITICAL TRANSPARENCY NOTE: The 137 → 86 antibody filtering procedure is NOT fully documented in Sakhnini et al. (2025).

What Novo Explicitly Documents

From Sakhnini et al. (2025), Section 4.1 and Table 2:

• —ELISA with 6 ligands: 0 flags = specific, 1-3 flags = mild (excluded), ≥4 flags = non-specific
• —Result: 137 → 116 antibodies (21 with ELISA 1-3 removed)
• —Figure S14A shows: 86 antibodies with confusion matrix [[40, 17], [10, 19]]

What We Reverse-Engineered

The paper does NOT document how they go from 116 → 86 antibodies. We reverse-engineered this step:

1. 1.Reclassify 7 antibodies (specific → non-specific) based on:
2. 2.Tier A: PSR > 0.4 (bimagrumab, bavituximab, ganitumab)
3. 3.Tier B: Tm < 60°C (eldelumab)
4. 4.Tier C: High clinical ADA rate (infliximab)
5. 5.Tier D: Chromatography flags (HIC > 11.7): lebrikizumab, galiximab

1. 1.Remove 30 specific antibodies with highest PSR scores (AC-SINS as tiebreaker)

Result: 57 specific + 29 non-specific = 86 antibodies → EXACT PARITY

Confidence Level

• —High confidence: The final result matches Novo's published confusion matrix exactly
• —Uncertainty: We don't know if Novo used this exact procedure — only that it reproduces their results
• —Alternative pairs exist: Two other antibody pairs also produce exact parity (documented in repository)
• —Our choice is principled: lebrikizumab + galiximab share the same flag type (chromatography), enabling a single methodologically consistent rule

Dataset Structure

Data Instances

{
  "id": "trastuzumab",
  "vh_sequence": "EVQLVESGGGLVQPGGSLRLSCAASGFNIKDTYIHWVRQAPGKGLEWVARIYPTNGYTRYADSVKGRFTISADTSKNTAYLQMNSLRAEDTAVYYCSRWGGDGFYAMDYWGQGTLVTVSS",
  "label": 0.0
}

Data Fields

Data Splits

Note: This dataset is used exclusively as a test set for models trained on the Boughter dataset.

Dataset Creation

Curation Rationale

This dataset was created to:

1. 1.Benchmark reproducibility: Verify independent reproduction of Novo Nordisk's published results
2. 2.Clinical relevance: Test generalization from mouse antibodies (training) to human clinical antibodies (testing)
3. 3.Scientific rigor: Document exactly what preprocessing was applied and what remains uncertain

Source Data

Original Data Collection

From Jain et al. 2017:

• —137 antibodies that reached Phase 2 clinical trials or FDA approval
• —Biophysical measurements: ELISA polyreactivity, PSR, AC-SINS, HIC, SMAC, thermal stability, and more
• —Published supplement files: SD01 (metadata), SD02 (sequences), SD03 (biophysical data)

Original Files:

• —jain-pnas.1616408114.sd01.xlsx — Antibody metadata
• —jain-pnas.1616408114.sd02.xlsx — VH and VL sequences
• —jain-pnas.1616408114.sd03.xlsx — Biophysical measurements (PSR, HIC, AC-SINS, etc.)
• —Private_Jain2017_ELISA_indiv.xlsx — Per-antigen ELISA flag data (provided by Adimab: T. Sun, Y. Xu; not in original PNAS paper)

Preprocessing Pipeline

Final: 86 antibodies (57 specific, 29 non-specific)

Implementation note: preprocessing/jain/step2_preprocess_p5e_s2.py applies Tier D after the PSR/AC-SINS removal step. This is equivalent because lebrikizumab and galiximab have PSR=0.0 and are not selected by the top-30 PSR/AC-SINS removal criterion.

P5e-S2 + Tier D Method Details

Step 2: ELISA Filtering (Novo Documented)

Antibodies classified by ELISA flag count:

• —0 flags → Specific (label=0) — INCLUDE
• —1-3 flags → Mildly polyreactive — EXCLUDE
• —≥4 flags → Non-specific (label=1) — INCLUDE

Result: 137 → 116 antibodies

Step 3: Reclassification (Reverse-Engineered)

Reclassification tiers (7 antibodies):

Tier D rationale: Both antibodies have elevated HIC retention times (chromatography flags) from Jain SD03 public data. HIC measures surface hydrophobicity, which directly correlates with non-specific binding (“stickiness”).

Step 4: PSR/AC-SINS Removal (Reverse-Engineered)

Remove the 30 specific antibodies with highest PSR scores (AC-SINS as secondary sort).

This yields the Novo target label distribution: 57 specific / 29 non-specific.

Annotations

Annotation Process

1. 1.ELISA Labels: Binary labels from per-antigen ELISA flags aggregated as total flag count (0 vs ≥4)
2. 2.Reclassification: Based on public biophysical data from Jain SD03 (Tiers A-D)
3. 3.PSR/AC-SINS Removal: Remove 30 highest-PSR specific antibodies

Who are the annotators?

• —Original ELISA assays: Tingwan Sun and Yingda Xu (Adimab, LLC) — see Jain et al. 2017
• —Per-antigen ELISA data: Provided by Adimab (M. Vásquez, T. Sun, Y. Xu) with permission for open research use
• —Preprocessing methodology: Based on Sakhnini et al. 2025 (Novo Nordisk & University of Cambridge)
• —Reverse-engineering & Tier D: The-Obstacle-Is-The-Way (reproducing Novo methodology)

Personal and Sensitive Information

This dataset contains clinical-stage antibody sequences from published sources. All sequences are from therapeutic antibodies that have been disclosed in scientific literature and patent applications. No personal or patient information is included.

Benchmark Results

Exact Novo Parity Achieved

Interpretation:

• —TN (True Negatives): 40 specific correctly predicted as specific
• —FP (False Positives): 17 specific incorrectly predicted as non-specific
• —FN (False Negatives): 10 non-specific incorrectly predicted as specific
• —TP (True Positives): 19 non-specific correctly predicted as non-specific

Considerations for Using the Data

Social Impact of Dataset

This dataset enables:

• —Benchmarking antibody developability prediction models
• —Evaluating cross-species generalization (mouse training → human testing)
• —Reproducibility verification of published ML results

Discussion of Biases

1. 1.Selection Bias: Only Phase 2+ clinical antibodies included (survival bias toward "good" candidates)
2. 2.Species Transfer: Models trained on mouse antibodies may not optimally transfer to human antibodies
3. 3.Assay Bias: ELISA panel may not capture all forms of non-specificity
4. 4.Reverse-Engineering Uncertainty: Tier D reclassification is inferred, not documented by Novo

Other Known Limitations

1. 1.VH Only: This export contains only heavy chain sequences; light chains available in full metadata file
2. 2.Small Size: 86 antibodies is small for deep learning; primarily useful for benchmarking
3. 3.Preprocessing Uncertainty: The 116 → 86 step is reverse-engineered (documented above)
4. 4.Imbalanced Classes: 66% specific vs 34% non-specific

Additional Information

Dataset Curators

• —Original Dataset: Jain et al. 2017 (Adimab, LLC & MIT)
• —Per-antigen ELISA Data: Tingwan Sun, Yingda Xu, Maximiliano Vásquez (Adimab, LLC)
• —Preprocessing Methodology: Laila I. Sakhnini, Daniele Granata et al. (Novo Nordisk)
• —Reverse-Engineering & This Preprocessing: The-Obstacle-Is-The-Way (Hugging Science)

Licensing Information

Jain et al. (2017) is published in PNAS (open access). The biophysical data in SD03 is publicly available from the PNAS supplementary materials. This Hugging Face export is distributed under the MIT license; please retain upstream attribution/citations (papers + repository).

Citation Information

If you use this dataset, please cite the original paper, the Novo Nordisk methodology paper, and this repository:

@article{jain2017biophysical,
  title={Biophysical properties of the clinical-stage antibody landscape},
  author={Jain, Tushar and Sun, Tingwan and Durand, St{\'e}phanie and Hall, Amy and Houston, Nga Rewa and Nett, Juergen H and Sharkey, Beth and Bobrowicz, Beata and Caffry, Isabelle and Yu, Yao and Cao, Yuan and Lynaugh, Heather and Brown, Michael and Baruah, Hemanta and Gray, Laura T and Krauland, Eric M and Xu, Yingda and V{\'a}squez, Maximiliano and Wittrup, K Dane},
  journal={Proceedings of the National Academy of Sciences},
  volume={114},
  number={5},
  pages={944--949},
  year={2017},
  publisher={National Academy of Sciences},
  doi={10.1073/pnas.1616408114}
}

@article{sakhnini2025prediction,
  title={Prediction of Antibody Non-Specificity using Protein Language Models and Biophysical Parameters},
  author={Sakhnini, Laila I. and Beltrame, Ludovica and Fulle, Simone and Sormanni, Pietro and Henriksen, Anette and Lorenzen, Nikolai and Vendruscolo, Michele and Granata, Daniele},
  journal={bioRxiv},
  year={2025},
  month={May},
  publisher={Cold Spring Harbor Laboratory},
  doi={10.1101/2025.04.28.650927},
  url={https://www.biorxiv.org/content/10.1101/2025.04.28.650927v1}
}

Contributions

Thanks to:

• —Jain et al. for publishing the original clinical antibody biophysical data
• —Adimab (T. Sun, Y. Xu, M. Vásquez) for providing per-antigen ELISA data with permission for open research
• —Novo Nordisk (Sakhnini et al.) for publishing their methodology, enabling independent replication
• —[The-Obstacle-Is-The-Way](https://github.com/The-Obstacle-Is-The-Way) for reverse-engineering and documenting the complete preprocessing pipeline

Alternative Parity Pairs (Documented for Transparency)

Three antibody pairs produce exact Novo parity. We chose lebrikizumab + galiximab, but alternatives are documented:

Why we chose lebrikizumab + galiximab:

1. 1.Both share the same flag type (chromatography/HIC)
2. 2.Enables a single methodologically consistent rule
3. 3.HIC measures surface hydrophobicity — directly related to non-specific binding
4. 4.Would be flagged by standard QC regardless of confusion matrix outcome

Full rationale: docs/bugs/jain_parity_decision.md in repository

Version: 1.0.0 Last Updated: 2025-12-16 Maintainer: Hugging Science Organization

<!-- Last verified: 2025-12-17T05:03:09Z -->