sonia12138/STLBench

STLBench

STLBench is a function-level dataset for C++ STL container type recovery from stripped binaries. Each record pairs a function's ground-truth C++ source with its stripped-binary disassembly, control-flow graph (CFG), Ghidra pseudo-code, and a multi-label set of STL container types used by the function.

Anonymous release for double-blind review. Authors and project repository will be revealed after the review period.

Summary

• —Records: 14,884 (3,894 unique source functions × 4 optimization levels, with per-opt extraction failures)
• —Source pool: 4,600 selected solutions from CodeContests (DeepMind, CC BY 4.0)
• —Toolchain: g++ + libstdc++, x86-64 Linux ELF
• —Optimization levels: O0, O1, O2, O3
• —Label space (6 classes): vector, map, pair, set, string, queue (multi-label; array excluded as it is undetectable post-compilation)
• —Splits: train / validation / test = 11,914 / 1,487 / 1,483 records, partitioned by problem_name (no problem leakage across splits)

Schema

Loading

from datasets import load_dataset

ds = load_dataset("STLBench/STLBench")
print(ds)
# DatasetDict({
#   train:      Dataset({features: [...], num_rows: 11914})
#   validation: Dataset({features: [...], num_rows: 1487})
#   test:       Dataset({features: [...], num_rows: 1483})
# })

ex = ds["train"][0]
print(ex["container_labels"], ex["opt"], len(ex["stripped_asm"]))

Construction Pipeline

The corpus is built by a stripped-binary extraction pipeline that, for each selected solution:

1. 1.Source filtering — pick the longest function in the solution that (a) is not main, (b) does not call any other user-defined function in the same source file, and (c) has at least 5 source lines.
2. 2.Compile — g++ -g -O{0,1,2,3} against libstdc++, producing four binaries per solution.
3. 3.Address resolution — nm -n on the unstripped binary recovers [start, next_sym) for the target function.
4. 4.Strip — copy and strip the binary to remove symbol tables.
5. 5.CFG extraction — angr.CFGFast on the stripped binary, anchored at the rebased start address.
6. 6.Disassembly — objdump -d on the stripped binary, sliced to the target's address range.
7. 7.Ghidra pseudo-code — Ghidra headless analyzer on the stripped binary, with the target function located via a multi-fallback resolver.
8. 8.Multi-label assignment — source-level clang-AST parse identifies STL container types appearing in the target function body.

Records that fail at any stage are dropped (overall pipeline retention 81%; per-opt success 93.5–98.6%).

The construction pipeline source code will be released after de-anonymization.

Splits

Splits are made at the problem level (problem_name), not at the function or record level. This matches the leakage-prevention protocol used in our experiments and prevents near-duplicate solutions of the same problem from spanning train/val/test.

• —Seed: 44
• —Ratio: 80 / 10 / 10 (train / validation / test) over unique problem_name values
• —Determinism: random.Random(seed).shuffle(sorted(problem_names))

Intended Uses

• —Function-level STL container type recovery from stripped C++ binaries (multi-label classification)
• —Pseudo-code refinement with type-grounded hints (LLM input: ghidra_pseudo + label hints; LLM target: func)
• —Binary-to-source code generation (input: stripped_asm or ghidra_pseudo; target: func)
• —Cross-optimization-level transfer studies (the four opt levels are aligned per problem_name)

Limitations

• —Single toolchain: g++ / libstdc++ / x86-64 only. Cross-compiler (clang++/libc++) and cross-architecture (ARM, RISC-V) generalization is not covered.
• —`array` excluded: std::array of POD types compiles to indistinguishable stack memory and cannot be recovered from binary alone.
• —Class imbalance: map, pair, set, queue have far fewer positive records than string and vector.
• —Ghidra coverage: ~3% of records have null ghidra_pseudo due to upstream Ghidra failures.
• —No runtime tests: unlike HumanEval-C++, CodeContests problems are not paired with executable unit tests in this release.

License & Attribution

This dataset is released under CC BY 4.0, inheriting the license of the upstream CodeContests source code (DeepMind, CC BY 4.0). Users must preserve attribution to CodeContests when redistributing.

The compiled artifacts (stripped_asm, cfg_dot, ghidra_pseudo) are derivative works of CodeContests source under the same license.

Citation

@misc{stlbench2026,
  title  = {{STLBench}: A Stripped-Binary Dataset for {STL} Container Type Recovery},
  author = {Anonymous},
  year   = {2026},
  note   = {Anonymized for review.}
}

Contact

Anonymous during the review period.