CoolFace
Apppublic

shiqiao123/umlip-dft-preopt-leaderboard

sourceHugging Facecc-by-4.0updated 2mo agoView on Hugging Face
0likes
App README

uMLIP DFT pre-optimizer leaderboard

Six universal machine-learning interatomic potentials, ranked as pre-optimizers for DFT: each one pre-relaxes a perturbed crystal under one frozen protocol, the pre-relaxed structure is handed to a real VASP relaxation that is run to its endpoint, and the potential is judged by what that continuation does - ionic steps and SCF iterations spent, whether it converges, and whether it lands in the same energy basin as a from-scratch DFT relaxation of the same input.

A DFT-free Pre-optimization Quality Index (PQI), calibrated on the 40-structure bulk subset and then applied frozen, decides which handoffs are safe to accept. The headline column is acceptance at the deployed operating point, PQI >= 70 - which is a decision rate, not an accuracy.

The census, stated honestly

200 structures are runnable. 159 have a converged from-scratch DFT reference. 148 were scored in the paper (40 in-domain + 108 transfer). These three numbers are all correct and they are not the same number: the readiness index needs no DFT, so every relaxed structure carries a score; only the 159 with a converged reference can be scored against a DFT endpoint; and only 148 of those had converged when the scoring set was frozen. Every leaderboard column is restricted to the 148 so that all rows are measured on the same structures. Nothing here implies that 200 structures have references.

What this Space does

It is a static Space: one HTML file, no server, no build step and no runtime. The page renders the leaderboard - a headline table with the subset chips and sortable metric columns, a per-domain breakdown of the transfer set, the cost-benefit plane, and the submission instructions - entirely in the browser.

The data contract is unchanged. The page fetches leaderboard.csv from the dataset repo on load and uses it when it arrives; the copy embedded in index.html (the same bytes as data/leaderboard.csv) renders the table immediately and remains in use if the fetch fails. A line under the table always states which one is showing. A new row arrives as a Pull Request against the dataset repo and this page follows it on the next reload.

FileWhat it is
index.htmlthe whole page - markup, style, script and an embedded copy of the three tables
SUBMISSION.mdhow to get your own row added; the same text is rendered in the Submit tab
build_domain_table.pyregenerates the per-domain table from the dataset, and refuses to write unless it reproduces leaderboard.csv
data/the tables as separate CSV files - see data/README.md for provenance

Add your own potential

Write one get_calculator() function, run four commands from the GitHub harness on the full 200-structure suite, open a Pull Request on the dataset repo adding one row plus your scores file. Full instructions: `SUBMISSION.md`. A submission is listed once its numbers reproduce from the submitted scores file; verification is manual.

Links

  • —Dataset - `shiqiao123/umlip-dft-preopt`: seven configs, the 200 input structures, the 159 converged DFT references, and the measurement tables behind every number here.
  • —Code - `ShiQiaoL/umlip-dft-preopt`: relaxation drivers, the standalone VASP convergence parser, the scoring analyses, the figure builder and the benchmark harness.
  • —Paper - "Benchmarking universal machine learning interatomic potentials as DFT pre-optimizers" (Liu, Wang, Yang, Shen; npj Computational Materials, submitted 2026).

Licensing

This Space is CC BY 4.0, matching the dataset it renders. The harness code is MIT on GitHub. The benchmark's input structures derive from Materials Project (CC BY 4.0, attribution required) and NIST JARVIS-DFT parents, plus heterostructures generated in that work; per-structure provenance and the full attribution are on the dataset card. No model checkpoint is redistributed here or anywhere in the project.