Datasetpublic

plawanrath/StableHLO-Spec-30

StableHLO-Spec-30 Hand-authored NL→StableHLO pairs across 10 op families (n=30). Composition Instances: 30 Format: one JSON record per line in data/test.jsonl Schema: fields = dialect, difficulty, id, mlir, nl, notes Verifier: stablehlo-opt v1.4.0 (upstream truth) and iree-compile --compile-to=input (substitute, 50/50 concordant on a stratified n=50 sample) License: Apache-2.0 (SPDX: Apache-2.0). No third-party IP restrictions. Loading from… See the full description on the dataset page: https://huggingface.co/datasets/plawanrath/StableHLO-Spec-30.

sourceHugging Faceapache-2.0updated 5mo agoView on Hugging Face

0likes16downloads

Dataset Card

StableHLO-Spec-30

Hand-authored NL→StableHLO pairs across 10 op families (n=30).

Composition

—Instances: 30
—Format: one JSON record per line in data/test.jsonl
—Schema: fields = dialect, difficulty, id, mlir, nl, notes
—Verifier: stablehlo-opt v1.4.0 (upstream truth) and iree-compile --compile-to=input (substitute, 50/50 concordant on a stratified n=50 sample)
—License: Apache-2.0 (SPDX: Apache-2.0). No third-party IP restrictions.

Loading

python

from datasets import load_dataset
ds = load_dataset("plawanrath/StableHLO-Spec-30", split="test")
print(ds[0])

Each record is a self-contained natural-language→MLIR pair; verify-valid pass-rate under the dialect's verifier is the primary evaluation metric.

Source format

The JSONL file at data/test.jsonl is the canonical HuggingFace interface. MLCommons Croissant 1.0 metadata (croissant.json) ships alongside the release.

Datasheet

Key points (full Gebru-style datasheet ships with the dataset archive):

—All reference MLIR programs are verifier-clean at the time of release.
—Hand-authored (no crowdsourcing, no LLM-authored references).
—Test-only — fine-tuning on these benchmarks contaminates future evaluation and is explicitly out of scope.

License

Apache-2.0. See LICENSE.