patched-codes/static-analysis-eval
A dataset of 76 Python programs taken from real Python open source projects (top 100 on GitHub), where each program is a file that has exactly 1 vulnerability as detected by a particular static analyzer (Semgrep), used in the paper Patched MOA: optimizing inference for diverse software development tasks. OpenAI used the synth-vuln-fixes and fine-tuned a new version of gpt-4o is now the SOTA on this benchmark. More details and code is available from their repo. More details on the benchmark… See the full description on the dataset page: https://huggingface.co/datasets/patched-codes/static-analysis-eval.
Add link to paper (#3)
Update README.md
Upload o1-mini-0-shot_semgrep_1.85.0_20240913_155514.log
Update README.md
Update README.md
Update README.md
Upload openai-static-analysis-eval.png
Update README.md
Update README.md
Upload 3 files
Update README.md
Upload 2 files
Upload dataset
Upload dataset
Update README.md
Upload 2 files
Update README.md
Update README.md
Update README.md
Update README.md
Upload 6 files
Update README.md
Update README.md
Update README.md
Upload _script_for_gen.py
Update README.md
Upload oracle-0-shot_semgrep_1.85.0_20240820_174931.log
Update README.md
Update README.md
Update README.md
Update README.md
Update README.md
Upload _script_for_eval.py
Update README.md
Update README.md
Upload ft_gpt-4o-2024-05-13_patched_patched-4o_9xp8XOM9-0-shot_semgrep_1.85.0_20240819_075205.log
Update README.md
Upload 2 files
Upload oracle-0-shot_semgrep_1.85.0_20240819_022711.log
Upload gpt-4o-0-shot_semgrep_1.85.0_20240819_015355.log
Update README.md
Upload _script_for_eval.py
Upload gpt-4o-mini-1-shot-sim_semgrep_1.85.0_20240819_013810.log
Upload gpt-4o-mini-3-shot_semgrep_1.85.0_20240818_234709.log
Update README.md
Update README.md
Upload _script_for_eval.py
Update README.md
Upload _script_for_eval.py
Upload 2 files