stephenjun8192/chemberta-zinc-sparse50

ChemBERTa-zinc Sparse 50% — PharmaCore

A 50% magnitude-pruned version of seyonec/ChemBERTa-zinc-base-v1 optimized for efficient molecular encoding on Apple Silicon.

Why This Model?

Use Case

Molecular encoder in the PharmaCore drug discovery pipeline:

• —Encode SMILES strings into dense embeddings for drug-target scoring
• —Molecular similarity computation for drug repurposing
• —Drug-likeness assessment and ADMET property prediction
• —Runs entirely on consumer Apple Silicon hardware (M1/M2/M3/M4)

Usage

from transformers import AutoModel, AutoTokenizer
import torch

model = AutoModel.from_pretrained("stephenjun8192/chemberta-zinc-sparse50")
tokenizer = AutoTokenizer.from_pretrained("seyonec/ChemBERTa-zinc-base-v1")

# Encode a drug molecule (Erlotinib — EGFR inhibitor)
smiles = "COCCOc1cc2ncnc(Nc3cccc(C#C)c3)c2cc1OCCOC"
inputs = tokenizer(smiles, return_tensors="pt", padding=True, truncation=True)

with torch.no_grad():
    outputs = model(**inputs)
    embedding = outputs.last_hidden_state.mean(dim=1)  # [1, 768]

print(f"Embedding shape: {embedding.shape}")

Sparsification Method

• —Technique: Global magnitude pruning (unstructured)
• —Sparsity: 50% of all weight parameters set to zero
• —Layers pruned: All linear layers (attention Q/K/V/O, FFN)
• —Validation: Cosine similarity of embeddings vs dense model ≥ 0.973
• —Training data: Pre-trained on 100K ZINC molecules (SMILES)

Benchmarks (Apple M4 Mac mini, 16GB)

Part of PharmaCore

PharmaCore — the first AI drug discovery platform that runs entirely on a MacBook. No cloud GPUs, no API keys, no data leaves your machine.

Citation

@software{pharmacore2026,
  title={PharmaCore: Apple Silicon-Native AI Drug Discovery},
  author={Stephen Wu},
  year={2026},
  url={https://github.com/reacherwu/PharmaCore}
}