dtsdts/multifidelity-dft-deltas

Multi-fidelity DFT delta corrections for crystal structures

Dataset Description

This dataset contains ~60,000 crystal structure relaxations from the NOMAD repository, computed with the all-electron DFT code FHI-aims at multiple levels of numerical precision (basis set size × k-point density). For each pair of a low-fidelity and a high-fidelity calculation on the same structure, we provide the delta corrections — the difference in key physical observables between the two settings.

The dataset was constructed to train and evaluate machine-learning models that predict the correction needed to "upgrade" a cheap DFT calculation to a more expensive one, without running the expensive calculation.

The default configuration is now legacy_deduplicated_leakage_safe_v1. It removes redundant exact copies, excludes conflicting duplicate groups, and uses connected components of ICSD identity and exact starting-structure hash to prevent leakage between train, validation, and test. The original 59,235-row release remains available as the legacy_original configuration for reproducibility.

Key quantities

Files

Loading Examples

Tabular data (Parquet)

import pandas as pd

df = pd.read_parquet("data.parquet")
print(df.shape)          # (59235, 175)
print(df["basis_size"].value_counts())
print(df["delta_total_energy_per_atom"].describe())

Recommended leakage-safe configuration

from datasets import load_dataset

dataset = load_dataset(
    "dtsdts/multifidelity-dft-deltas",
    "legacy_deduplicated_leakage_safe_v1",
)
print(dataset)
# train: 39,951; validation: 4,940; test: 5,217

The hosted partitions retain the split column used to create them. The historical partition label is preserved separately as source_split.

Graph data (PyTorch / PyG)

import torch

graphs = torch.load("graph_data.pt", map_location="cpu")
# graphs is a list of torch_geometric.data.Data objects
g = graphs[0]
print(g.x.shape)          # atom features
print(g.edge_index.shape) # connectivity
print(g.pos.shape)        # fractional / Cartesian positions
# g.split  →  "train" | "val" | "test"

Column Dictionary

Identifiers & DFT settings

Primary delta targets

Structure geometry (low-fidelity baseline)

Elemental features (statistics over atoms in unit cell)

Columns follow the pattern {stat}_{feature} where stat ∈ {max, min, mean, mad}:

Dataset Statistics

• —Original release: 59,235 rows
• —Recommended deduplicated release: 50,108 rows
• —Recommended split: 39,951 train / 4,940 validation / 5,217 test
• —Exact duplicate copies removed: 6,200
• —Conflicting duplicate rows excluded: 971 rows across 360 groups
• —Cross-partition leakage: 0 shared ICSD numbers and 0 shared exact structure hashes
• —Unique structures: ~14,500
• —Basis sizes: minimal, standard, tier1, tier2
• —k-point densities: 2, 4, 8
• —Elements covered: H through Bi (excluding lanthanides/actinides; a small number of heavy-element structures are filtered)
• —Split method: deterministic connected components of ICSD number and exact starting-structure hash

Deduplication policy

An exact duplicate key combines the exact starting-structure hash with the normalized physical FHI-aims setting. For identical copies, the row with the lexicographically smallest padded source-row identifier is retained. If rows with the same key disagree in their target values, the entire conflicting group is excluded rather than selecting or averaging a label. See the included audit.json for the complete reconciliation and SHA-256 hashes.

Citation

If you use this dataset, please cite:

@inproceedings{speckhard2026multifidelity,
  title     = {Multi-fidelity {DFT} delta corrections for crystal structures},
  author    = {Speckhard, Daniel and others},
  booktitle = {ICML 2026 Workshop on Machine Learning for Materials},
  year      = {2026},
}

License

Creative Commons Attribution 4.0 International (CC BY 4.0)

Training Code

The GNN and Random Forest training code used with this dataset is available at: https://gitlab.mpcdf.mpg.de/dansp/errorbar_modelling