Datasetpublic

HUBioDataLab/SELFormerMM

SELFormerMM Multimodal molecular representation data for SELFormerMM — an extension of SELFormer that aligns four complementary views of a molecule in a shared embedding space: SELFIES sequences, 3D/structural graphs, textual descriptions, and knowledge-graph context, trained with multimodal supervised contrastive learning on ~2.9M molecules. Paper: SELFormerMM: multimodal molecular representation learning via SELFIES, structure, text, and knowledge graph integration… See the full description on the dataset page: https://huggingface.co/datasets/HUBioDataLab/SELFormerMM.

sourceHugging Facegpl-3.0updated 21d agoView on Hugging Face

0likes129downloads

Dataset Card

SELFormerMM

Multimodal molecular representation data for SELFormerMM — an extension of SELFormer that aligns four complementary views of a molecule in a shared embedding space: SELFIES sequences, 3D/structural graphs, textual descriptions, and knowledge-graph context, trained with multimodal supervised contrastive learning on ~2.9M molecules.

Paper: [SELFormerMM: multimodal molecular representation learning via SELFIES, structure, text, and knowledge graph integration](https://doi.org/10.1093/bioinformatics/btag451) (Bioinformatics, 2026)

Code: https://github.com/HUBioDataLab/SELFormerMM

This repository bundles everything needed to reproduce or build on SELFormerMM:

Component	What it is
`pretraining_datasets/`	2,854,815 ChEMBL v36 molecules with precomputed structure, text and KG embeddings, plus the heterogeneous KG itself
`finetuning_datasets/`	9 downstream MoleculeNet benchmarks (BACE, BBBP, HIV, SIDER, Tox21, ESOL, FreeSolv, Lipophilicity, PDBbind), each with a metadata CSV and an aligned `.npz` of the three non-SELFIES modalities
`models/SELFormerMM/`	The multimodal RoBERTa backbone pretrained on the data above
`models/finetuned/`	One fine-tuned checkpoint per downstream task
`models/DMGI/`	The DMGI encoder used to produce the knowledge-graph embeddings

Every artifact in this repository is row-aligned. Row i of a metadata CSV describes the same molecule as row i of every .npy / .npz array that accompanies it.

A molecule that has no description, no KG node, or a structure that failed RDKit validation gets a zero vector in that modality rather than being dropped. Downstream code is expected to treat an all-zero row as "modality missing".

If you regenerate any modality yourself, you must preserve this ordering or the model will silently receive mismatched inputs.

Pretraining data

pretraining_datasets/ — 2,854,815 drug-like molecules drawn from ChEMBL v36.

File	Shape / size	Description
`pretraining_dataset_meta.csv`	2,854,815 rows, 1.7 GB	Identifiers, structures and raw descriptions
`graph_embeddings.npy`	`(2854815, 512)` float32	Uni-Mol `[CLS]` representations
`text_embeddings.npy`	`(2854815, 768)` float32	SciBERT mean-pooled description embeddings
`kg_embeddings.npy`	`(2854815, 128)` float32	DMGI node embeddings
`selformermm_kg_heterodata.pt`	2.2 GB	The CROssBAR v2 subgraph as a PyTorch Geometric `HeteroData` object

`pretraining_dataset_meta.csv` columns

Column	Description
`selfies`	SELFIES string — the model's primary input
`chembl_id`	ChEMBL identifier, e.g. `CHEMBL153534`
`canonical_smiles`	Canonical SMILES from ChEMBL
`standard_inchi`	InChI
`standard_inchi_key`	InChIKey
`cid`	PubChem CID, when a mapping exists
`Description`	Textual description mapped from the M3-20M dataset
`kg_compound_node_idx`	Index of this compound's node in `selformermm_kg_heterodata.pt`, when present in the CROssBAR v2 subgraph

Modality coverage

Modality	Molecules with a non-zero vector	Coverage	Source
SELFIES	2,854,815	100.00%	ChEMBL v36 SMILES, converted with `selfies`
Structure (Uni-Mol)	2,854,734	100.00%	81 molecules failed RDKit validation and were excluded
Text (description)	492,702	17.26%	Mapped to M3-20M
Knowledge graph	725,908	25.43%	CROssBAR v2 subgraph

The knowledge graph

selformermm_kg_heterodata.pt is a focused subgraph of CROssBAR v2 (which itself integrates 34 heterogeneous biomedical sources), restricted to the node and edge types most directly tied to molecular properties — compounds, proteins, drugs and genes. It contains 1,402,102 nodes and 4,424,830 relationships. Node indices referenced by kg_compound_node_idx are into the Compound node store, whose mapping attribute keys are of the form chembl:CHEMBL6206.

How the embeddings were produced

Every non-SELFIES encoder is a frozen pretrained checkpoint — only the SELFIES encoder and the projection networks were trained.

—Structure — 512-d. Uni-Mol (unimol_tools.UniMolRepr, data_type="molecule", hydrogens retained) applied to the canonical SMILES; the cls_repr vector is taken as the molecule representation.
—Text — 768-d. SciBERT (allenai/scibert_scivocab_uncased), max length 512, mean-pooling over the last hidden state of the Description field.
—Knowledge graph — 128-d. A DMGI (Deep Multiplex Graph Infomax) encoder — one GCNConv per relation type with a bilinear discriminator against a per-relation graph summary — trained on selformermm_kg_heterodata.pt. Per-relation node embeddings are averaged to give the final vector. The checkpoint is at models/DMGI/dmgi_model.pt.

All three matrices are mean-centered and L2-normalized over the non-zero rows; zero rows (missing modality) are left untouched so they stay exactly zero.

Fine-tuning datasets

finetuning_datasets/ — nine downstream benchmarks, each a directory containing <task>.csv and <task>_embs.npz. The .npz holds three arrays, graph (n, 512), text (n, 768) and kg (n, 128), row-aligned to the CSV and produced by the same encoders as the pretraining data.

Classification

Task	Molecules	Label column(s)	Notes
`bace`	1,513	`Class`	Binary — human β-secretase 1 inhibition. 691 positives (45.7%)
`bbbp`	2,039	`p_np`	Binary — blood–brain barrier permeability. 1,560 positives (76.5%)
`hiv`	41,127	`HIV_active`	Binary — HIV replication inhibition. 1,443 positives (3.5%)
`sider`	1,427	27 columns	Multi-label — adverse drug reactions by MedDRA system organ class
`tox21`	7,831	12 columns	Multi-label — nuclear-receptor and stress-response toxicity assays

Regression

Task	Molecules	Target column	Range (mean)
`esol`	1,128	`measured log solubility in mols per litre`	−11.60 … 1.58 (−3.05)
`freesolv`	642	`freesolv`	−25.47 … 3.43 (−3.80)
`lipo`	4,200	`lipo`	−1.50 … 4.50 (2.19)
`pdbbind_full`	9,880	`-logKd/Ki`	0.40 … 9.30 (6.24)

Besides its label column(s), every task CSV carries selfies, smiles, and the mapping columns standard_inchi_key, cid, Description, chembl_id, canonical_smiles, kg_compound_node_idx. esol additionally keeps the original MoleculeNet descriptor columns.

Per-task modality coverage

Percentage of rows with a non-zero vector in each modality:

Task	n	Graph	Text	KG	All four
`bace`	1,513	100.00%	7.40%	13.09%	2.12%
`bbbp`	2,039	100.00%	67.48%	2.70%	1.86%
`hiv`	41,127	99.98%	14.23%	0.78%	0.28%
`sider`	1,427	91.03%	80.66%	1.89%	1.40%
`tox21`	7,831	99.90%	76.76%	4.61%	3.75%
`esol`	1,128	100.00%	88.30%	3.55%	3.46%
`freesolv`	642	100.00%	91.59%	4.83%	4.67%
`lipo`	4,200	100.00%	32.19%	24.05%	5.24%
`pdbbind_full`	9,880	99.91%	23.66%	4.96%	1.47%

SELFIES coverage is 100% for every task.

Splits

The CSVs are shipped unsplit. The paper's protocol is 80/10/10 train/validation/test, with scaffold splitting for the binary classification tasks (BACE, BBBP, HIV) and random splitting for the multilabel and regression tasks. train_finetuning.py reproduces this via --use_scaffold, --train_frac/--val_frac/--test_frac and --seed; reported numbers are means over three seeds.

Models

`models/SELFormerMM/` — pretrained multimodal backbone

A RoBERTa encoder over SELFIES (12 layers, hidden size 768, 4 attention heads, vocabulary 800, max position 514), initialized from SELFormer, with three parallel projection MLPs that map the structure (512-d), text (768-d) and KG (128-d) vectors into the same 768-d space. Each projection expands and contracts the dimension through ×4 → ×6 → ×6 → ×4 → ×1 of the hidden size with LayerNorm + ReLU between layers. 246,245,376 trainable parameters in total.

Trained with SINCERE loss (τ = 0.07), a supervised extension of InfoNCE that accommodates multiple positive views per molecule. The three auxiliary encoders stay frozen; only the SELFIES encoder and the projections are updated. Includes the SELFIES BPE tokenizer.

`models/finetuned/<task>/`

One checkpoint per downstream task (bace, bbbp, esol, freesolv, hiv, lipo, pdbbind_full, sider, tox21). Each directory holds the task tokenizer plus model.pt, a checkpoint dict with backbone and head state dicts, loadable by predict.py in the code repository.

`models/DMGI/dmgi_model.pt`

The DMGI encoder checkpoint that generated kg_embeddings.npy.

Reported performance

Selected results from the paper, mean ± standard deviation over three random seeds. ROC-AUC for classification (higher is better), RMSE for regression (lower is better). See the paper for the full table and the unimodal/multimodal baselines.

Task	Metric	SELFormerMM
SIDER	ROC-AUC	0.751 ± 0.013
BACE	ROC-AUC	0.779 ± 0.021
BBBP	ROC-AUC	0.947 ± 0.005
HIV	ROC-AUC	0.788 ± 0.025
Tox21	ROC-AUC	0.845 ± 0.012
ESOL	RMSE	0.672 ± 0.060
FreeSolv	RMSE	1.070 ± 0.065
Lipophilicity	RMSE	0.624 ± 0.049
PDBbind	RMSE	1.310 ± 0.040

Usage

Browse a downstream task in the viewer

python

from datasets import load_dataset

bbbp = load_dataset("HUBioDataLab/SELFormerMM", "bbbp")

The named configs above expose the nine task CSVs. The embedding arrays are not part of these configs — datasets cannot align .npz files row-wise — so download them directly.

Load a task with all four modalities

python

import numpy as np, pandas as pd
from huggingface_hub import hf_hub_download

REPO = "HUBioDataLab/SELFormerMM"
meta = hf_hub_download(REPO, "finetuning_datasets/classification/bbbp/bbbp.csv", repo_type="dataset")
embs = hf_hub_download(REPO, "finetuning_datasets/classification/bbbp/bbbp_embs.npz", repo_type="dataset")

df = pd.read_csv(meta)
z = np.load(embs)
graph, text, kg = z["graph"], z["text"], z["kg"]

assert len(df) == len(graph) == len(text) == len(kg)      # row-aligned
has_kg = np.linalg.norm(kg, axis=1) != 0                  # zero vector == modality missing

Fine-tune from the pretrained backbone

bash

huggingface-cli download HUBioDataLab/SELFormerMM --repo-type dataset \
  --include "models/SELFormerMM/*" "finetuning_datasets/classification/bbbp/*" --local-dir ./selformermm

python train_finetuning.py \
  --model_path ./selformermm/models/SELFormerMM \
  --dataset_meta_csv ./selformermm/finetuning_datasets/classification/bbbp/bbbp.csv \
  --dataset_embs_npz ./selformermm/finetuning_datasets/classification/bbbp/bbbp_embs.npz \
  --task_type binary --label_column p_np --num_labels 1 \
  --use_scaffold 1 --epochs 50 --backbone_lr 1e-5 --head_lr 1e-4 \
  --save_dir ./runs/bbbp

Predict with a released fine-tuned checkpoint

bash

python predict.py \
  --model_dir ./selformermm/models/finetuned/bbbp \
  --input_meta_csv ./selformermm/finetuning_datasets/classification/bbbp/bbbp.csv \
  --input_embs_npz ./selformermm/finetuning_datasets/classification/bbbp/bbbp_embs.npz \
  --task_type binary --num_labels 1 --label_column p_np \
  --output_csv ./bbbp_predictions.csv

Pretrain on your own corpus

Supply a SELFIES CSV plus a .npy per modality with identical row ordering; substitute a zero matrix of the right shape for any modality you do not have. See train_pretraining.py and the generate_*_embeddings.py scripts in the code repository.

Citation

If you use this data, please cite:

bibtex

@article{ulusoy2026selformermm,
  title   = {SELFormerMM: multimodal molecular representation learning via SELFIES,
             structure, text, and knowledge graph integration},
  author  = {Ulusoy, Erva and Bostanc{\i}, {\c{S}}evval and Deniz, Bora Engin and Do{\u{g}}an, Tunca},
  journal = {Bioinformatics},
  volume  = {42},
  number  = {Supplement\_2},
  pages   = {btag451},
  year    = {2026},
  doi     = {10.1093/bioinformatics/btag451}
}

License

GNU General Public License v3.0 or later.

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

Data redistributed here remains subject to the terms of its original sources (ChEMBL, M3-20M, CROssBARv2, MoleculeNet, PDBbind).

Contact

HU Biological Data Science Lab — open an issue on the code repository.