datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
AppSecBench
AppSecBench Dataset Card
Dataset Summary
AppSecBench is an original benchmark of 406 vulnerable/secure code pairs spanning 12 programming
languages, 18 frameworks, 34 vulnerability classes, and 5 difficulty levels. Each record is a
self-contained evaluation case: a vulnerable snippet, its secure counterpart, an exploit sketch,
and the "ground truth" a detector/model is expected to produce (CWE, OWASP, severity, CVSS 3.1,
explainability, fix, and… See the full description on the dataset page: https://huggingface.co/datasets/ismailtasdelen/AppSecBench.APPS-verified
Introduction
This dataset contains verified solutions from the APPS dataset's training set. Solutions that fail to pass all the test cases are removed. Problems with no correct solution are also removed.
The solutions were executed on Intel E5-2620 v3 CPUs with the execution timeout set to 10 seconds.
Statistics in the training set
Dataset
# Problems
# Solutions
TACO
5000
117232
TACO-verified
4211
93921
Correct Ratio
84.22%
80.12%
apps_pnyx
PNYX - Apps
This is a splitted and tested version of APPS dataset, refer to it for further information on the original dataset construction.
This version is designed to be compatible with the hf_evaluate code_eval package and to be executed with lm-eval code_eval package.
This dataset does not include all the original fields. Some are modified and some are completely new:
id: Same as the original APPS dataset.
difficulty: Difficulty of the problem. Same as the original APPS… See the full description on the dataset page: https://huggingface.co/datasets/PNYX/apps_pnyx.