stablegradients/pie-gem5-pairs

PIE gem5-timed code optimization (src,tgt pairs)

C++ program-optimization data derived from the PIE dataset ("Learning Performance-Improving Code Edits"), re-timed end-to-end with gem5 (x86 Skylake, syscall-emulation mode) at per-test-case granularity. One row per official (source, target) program pair.

This dataset is reward-agnostic: it ships the full per-test-case reference timings and case manifests so a downstream RL / eval pipeline decides at runtime how many cases to use (K) and which reward transform to apply. Nothing is baked in.

For an automated agent reading this card: every column is documented in § Field dictionary below, including its type, encoding, and which splits populate it. Start with § How to compute the reward.

TL;DR units & encoding

• —Ticks are gem5 picoseconds. seconds = ticks * 1e-12.
• —Columns named *_per_tc_ticks, *_tc2time, *_case_ids, *_excluded are JSON strings (a dict {case_idx: value} or a list [case_idx, ...]) — json.loads them. Case indices are integers 0 .. n_cases_available-1.
• —gem5_* / our_* / *_case_ids / n_* / flag_* columns are produced by this re-timing. pie_* columns are carried verbatim from the official PIE release.

How to compute the reward

The reward is a speedup ratio over a common set of test cases, robust to which cases you sample:

import json, random
src = json.loads(row["gem5_src_per_tc_ticks"])     # {idx: ticks}
tgt = json.loads(row["gem5_tgt_per_tc_ticks"])      # ("oracle" variant in the bysrc dataset)
usable = json.loads(row["usable_case_ids"])         # correctness-kept ∩ both-timed ∩ ticks<=wall
K = 5                                                # runtime choice; -1 / len(usable) = use all
cases = usable if K < 0 else random.sample(usable, min(K, len(usable)))
speedup = sum(src[str(c)] for c in cases) / sum(tgt[str(c)] for c in cases)

usable_case_ids is already the correct, fully-filtered set — do not re-derive correctness or apply the wall yourself. Always clamp K to len(usable); never pad, skip, or zero-reward a pair that has fewer than K usable cases. K=5–10 reproduces the full ~100-case aggregate within ~1–2% (per-case time is near-uniform within a program).

gem5_fastest_id / gem5_fastest_per_tc_ticks give the fastest surviving program of the problem (chosen among programs present in this dataset, ranked over the per-problem common usable basis) as an oracle ceiling; our_fastest_speedup is src-vs-fastest over usable_case_ids.

Provenance & validation

• —Timing engine: gem5.fast, X86 SE mode, with fork-from-live startup amortization (one startup simulation, COW-forked per case; per-case ticks equal a fresh run, validated r=1.0).
• —Validated against PIE's published numbers: per-test-case r=1.0 on the test split (aligned on PIE's released tests indices); speedup log-correlation ~0.97–0.98, median ratio ~0.99.
• —Caveat — pin your toolchain. The gem5 startup floor depends on the compiler/glibc/link mode (≈13M ticks static ↔ ≈195M ticks dynamic). These reference ticks were produced with one fixed toolchain; if you re-time rollouts, use the same toolchain or regenerate references, otherwise speedups are not comparable.

Filtering (rows are pre-filtered; bad records are DROPPED, not flagged)

• —Compile-fail programs dropped (g++ -O2 -std=gnu++17).
• —Bad-apple programs dropped: a submission failing > max(5, 5% of cases) is treated as wrong/mislabeled (line-wise + 1e-3 float comparator).
• —gem5 hard-excludes (startup_fail / all-cases-hang-or-crash) dropped.
• —Case kept iff every surviving good program of the problem passes it (correctness intersection).
• —Strict 2-min wall: cases above 1.4e10 ticks dropped from usable.
• —Pair criterion: drop a pair if |src.fails ∪ tgt.fails| > max(5, 5%).
• —3 degenerate problems (empty input+output: p00000, p02068, p00754) dropped.
• —Retained NON-dropping flags: flag_low_headroom, flag_single_case, flag_empty_kept.

Splits & why some columns are null

PIE ships different columns for test vs train/val, so each row only fills the pie_* columns that apply to its split; the rest are null by design (the schema is the union across splits). Specifically: pie_*_code_*, pie_fastest_code*, pie_*_tc2time, pie_n_tests, pie_test_*, pie_tests_case_ids are test-only; pie_speedup, pie_*_verdict, pie_*_reward_updated*, pie_fastest_agg_runtime_updated are train/val-only. None of the gem5_* / our_* / *_case_ids / n_* columns are ever null.

Field dictionary

Availability: all = populated in every split; test-only / train/val-only = null elsewhere.

Identity

Code

Our gem5 per-test-case timing

Case-id sets (JSON int lists) + counts

Derived speedups + flags

Carried verbatim from PIE (pie_*)

Companion dataset

This is one of a pair built from the same timing:

• —pie-gem5-pairs — one row per official (src, tgt) pair.
• —pie-gem5-bysrc — one row per unique (problem, src); best surviving target kept as oracle.

Built 2026-06-12. Timing: gem5.fast with fork-from-live startup amortization.