hheiden/PubChem-124M-SMILES-SELFIES-InChI-IUPAC

PubChem-124M-Canonicalized-SELFIES-InChI-IUPAC

Dataset Summary

This dataset contains ~124 million chemical structures sourced from PubChem (as of Jan 2026), processed into a clean, machine-learning-ready Parquet format.

Unlike raw XML/JSON dumps or standard CSVs, this dataset provides a unified, tabular structure that joins multiple chemical identifiers and descriptors into a single sharded resource:

• —SMILES: Raw and RDKit-Canonicalized.
• —SELFIES: Pre-computed 100% robust molecular string representations.
• —InChI: Standard InChI strings (mapped from PubChem auxiliary files).
• —IUPAC: Preferred IUPAC names (mapped from PubChem auxiliary files).
• —Mass: Monoisotopic and Exact Mass information.

It is sharded into 124 files (~1M rows each) and globally shuffled to ensure I.I.D. distribution for streaming training.

Data Fields

Note: Missing values in SELFIES usually indicate structures that violate standard valence rules or cannot be represented in the SELFIES grammar. Rows with missing InChI or IUPAC values correspond to PubChem entries where that specific auxiliary data was not provided in the dump.

Usage

Loading with Hugging Face

from datasets import load_dataset

# Load the entire dataset (warning: large)
ds = load_dataset("hheiden/PubChem-124M-Canonicalized-SELFIES-InChI-IUPAC")

# Streaming (Recommended for training)
ds_stream = load_dataset("hheiden/PubChem-124M-Canonicalized-SELFIES-InChI-IUPAC", streaming=True)
for sample in ds_stream['train']:
    print(sample['SMILES'], sample['SELFIES'])

Filtering for Translation Tasks

This dataset is particularly powerful for chemical translation tasks (e.g., SMILES $\rightarrow$ IUPAC). You can filter for rows where your target logic is present:

# Filter for samples with both SELFIES and IUPAC names
ds = ds.filter(lambda x: x['SELFIES'] is not None and x['iupac'] is not None)

Processing Steps

1. 1.Source: Downloaded raw Compound data and auxiliary files (CID-InChI-Key.gz, CID-IUPAC.gz, CID-Mass.gz) from PubChem.
2. 2.Canonicalization: Processed all SMILES through RDKit to ensure validity and standard formatting.
3. 3.SELFIES: Converted Canonical SMILES to SELFIES representations.
4. 4.Sharding & Join: Globally shuffled the base SMILES into 124 shards, then joined auxiliary data (InChI, IUPAC, Mass) per shard using DuckDB to ensure atomic row alignment.

License

The original data is from PubChem (Public Domain/US Government Work). This collation and processing is released under CC0 1.0 Universal.

Raw data available here