Scandium-Labs/Scandium-Dataset

Dataset Card — Scandium-Dataset v1.0.0

Summary

Scandium-Dataset provides a harmonized, quality-scored foundation of DFT-computed structural and thermodynamic properties across 267,230 materials from Materials Project, OQMD, and JARVIS-DFT. It supports the early screening stage of battery materials discovery — filtering by phase stability, electronic structure, and structural family — before downstream property prediction (ionic conductivity, mechanical stability, electrochemical window) via AIMD, NEB, or targeted DFT.

Previous versions positioned this as a solid-state battery discovery dataset. v1.0.0 corrects that framing: the data covers thermodynamic screening, not ionic transport. See [CHANGELOG](CHANGELOG.md) for details.

• —Version: v1.0.0
• —Total entries: 267,230 (Gold: 96,242 | Validated: 140,382 | Raw: 30,108 | Experimental: 498)
• —Strict Gold: 56,966 entries (subset of Gold with all gates passed)
• —Battery-relevant subset: 82,925 entries | Electrolyte candidate subset: 41,665 entries (strict Gold)

File names on disk (`battery_candidate_subset_v1.json`, `solid_electrolyte_candidate_subset_v1.json`) — the v3 suffix was retired in v1.0.0-rc2 when the scope was clarified.

• —Storage: Typed Parquet (dataset/entries_v4_typed.parquet, 47 columns) with indexed lookup — 178 MB with proper int/float/bool types for HF Dataset viewer compat.
• —Schema: Full column dictionary at `SCHEMA.md` — names, types, units, null semantics, source mappings.
• —Preview: `dataset/entries_v4_typed_sample.parquet` — 1,000-row stratified sample (1.25 MB) for schema inspection without downloading the full dataset.
• —Experimental data: 498 OBELiX entries integrated (Therrien et al. 2025) as experimental_gold tier with measured Li-ion conductivity.
• —Transport proxy draft: 24,873 BVSE migration barriers computed (bvlain engine) at 23% Li/Na coverage — see "Roadmap" below for the gap this addresses.

Quick Start

# Inspect schema (no download needed)
import pyarrow.parquet as pq
schema = pq.read_schema("dataset/entries_v4_typed_sample.parquet")
for f in schema:
    print(f"{f.name}: {f.type}")

# Load full dataset
import pandas as pd
df = pd.read_parquet("dataset/entries_v4_typed.parquet")
print(df.shape, list(df.columns))

# Stream in batches (low memory)
pf = pq.ParquetFile("dataset/entries_v4_typed.parquet")
for batch in pf.iter_batches(batch_size=1000, columns=["formula","band_gap"]):
    df_chunk = batch.to_pandas()
    print(f"Loaded {len(df_chunk)} rows")

# Filter by source before loading
df_mp = pd.read_parquet("dataset/entries_v4_typed.parquet", filters=[("source","=","mp")])

# Load partitioned source
df_mp2 = pd.read_parquet("dataset/partitioned/source=mp/data.parquet")

# Load from chunk
df_chunk = pd.read_parquet("dataset/chunks/entries_v4_chunk_00.parquet")

See `SCHEMA.md` for full column dictionary with units and null semantics.

Included vs. Not Included

Suitable For / Not Suitable For

Suitable for: phase stability screening, structural family classification, band gap prediction, materials-informatics benchmarking, pretraining general property predictors on inorganic crystal structures, cross-source DFT property harmonization studies.

Not suitable for (without augmentation): direct ionic conductivity prediction, SSE performance ranking, electrochemical stability assessment. The dataset contains migration barrier proxies for 23% of Li/Na entries, but at ~75% skip rate due to bond-valence parameter coverage, these do not constitute a complete transport-property layer.

What the Data Actually Supports

Property coverage (verified against Parquet store)

SSE screening fields (stored in ssb_screening block)

Sources

License Warning

⚠️ This dataset is NOT uniformly licensed. Each entry carries its own license.

• —license: "CC-BY-4.0" → MP entries (commercial safe, 26.1%)
• —license: "CC0-1.0" → JARVIS entries (commercial safe, 9.6%)
• —license: "OQMD-noncommercial" → OQMD entries (non-commercial only, 64.3%)

See `LICENSE_BREAKDOWN.md`. A Commercial-Safe edition (MP+JARVIS, ~94,952 entries) is extractable via scripts/extract_commercial_safe_edition.py.

Tier System

Family Imbalance

The dataset skews heavily toward intermetallics (62.5%) and layered oxides (15.7%). Solid-electrolyte-relevant composition families are a small fraction:

If your target chemistry is garnets or sulfides specifically, usable Gold-tier entries number in the tens to low thousands — consider targeted acquisition (ICSD, structured literature extraction) before expecting ML models to generalize within these families.

BVSE Migration Barriers (Draft Quality)

Caveat: 73,900 skipped entries (~75% of attempted) reflect bond-valence parameter coverage, not a sampling gap. Entries are flagged with bvse_skip_reason. Engine: bvlain v0.25.1, validated against 7 known SSEs (5/7 pass within literature, 2 known-marginal outliers documented in KNOWN_ISSUES.md).

Roadmap to True SSE-Property Coverage

v1.0.0 provides thermodynamic and structural screening data. Planned extensions:

1. 1.CAVD re-computation against the Parquet store (currently 0% coverage; algorithm exists in scripts/compute_cavd_channel_dimensionality.py)
2. 2.JARVIS EaH re-computation against the Parquet store (currently 0% for 25,673 JARVIS entries; script exists in scripts/compute_jarvis_hull_energy.py)
3. 3.BVSE barrier expansion — re-run with lower --max-sites threshold and extended parameter table to reduce the 75% skip rate
4. 4.MLIP-NEB migration barriers for top-tier stable candidates (active development)
5. 5.DFT NEB validation for a curated set of halide/sulfide/garnet candidates
6. 6.Elastic tensor data from MP API for mechanical property validation (scaffold exists in scripts/compute_mechanical_properties.py)
7. 7.Experimental conductivity cross-references beyond OBELiX pilot

Benchmark

Frozen train/val/test splits at `dataset/splits/`. Four split types with RF+Ridge baselines in `MODEL_LEADERBOARD.md`:

GNN baselines (CGCNN, MEGNet, ALIGNN) are in progress.

Intended Use

• —Primary: Upstream materials screening — filtering by phase stability, electronic structure, and structural family
• —Secondary: Cross-source DFT property harmonization, materials-informatics benchmarking, pretraining structure-based property predictors
• —Not recommended for: Quantitative phase diagram construction (use MP/OQMD directly), SSE conductivity ranking (requires transport-property labels not in this dataset)

Related Datasets

• —**Scandium-Labs/solid-state-electrolyte-conductivity** — the Scandium Labs SSB electrolyte transport dataset: literature-verified ionic conductivity & activation energy labels (183 verified) across 11 electrolyte families, with cross-paper consensus and a gold benchmark. Use it downstream of this dataset, where the screening filter has already narrowed candidates to those needing transport-property prediction.

Maintenance

• —Version: v1.0.0
• —DOI: pending (Zenodo archival in progress)
• —Issue tracking: GitHub Issues
• —Contact: Scandium Labs