durinn/Durinn_Hacktoberfest_Retrospective

Durinn Hacktoberfest Retrospective Dataset

Author: Ryan Marinelli & Victor Strandmoe Project: Durinn — Scaling Vibe Coding Auditing Dataset Type: Security SFT (Supervised Fine-Tuning) Sources: Scanning GitHub Hacktoberfest 2025 Format: HuggingFace DatasetDict with train and validation splits

📌 Overview

This dataset provides security-focused training data derived from analyzing Hacktoberfest 2025 GitHub repositories before and after the event using Semgrep’s OWASP Top 10 ruleset.

It contains two types of supervised fine-tuning (SFT) examples:

1. 1.Repository-level summaries High-level assessments of how a repo’s security posture changed between before (Oct 1) and after (Nov 1) snapshots.

1. 1.Snippet-level findings Per-finding examples showing:
2. 2.Code before and after
3. 3.Git diff between snapshots
4. 4.Semgrep rule, severity, OWASP categories
5. 5.Whether the finding persisted or was fixed

These examples are formatted in instruction–response form for use in training LLMs to:

• —Detect insecure code
• —Explain vulnerabilities
• —Understand OWASP semantics
• —Recognize fixes in Git diffs
• —Generate more secure code patterns

This dataset powers Durinn, a pipeline for auditing "vibe code" with Semgrep + LLM security critics.

📂 Files Included

train/ and validation/

Both contain rows with:

🧠 Dataset Schema

Each row follows this schema:

{
  "id": "unique_example_id",
  "source": "repo | snippet",
  "prompt": "<instruction + context + code + diff>",
  "response": "<model answer>",
  "meta": {
    "type": "repo|snippet",
    "repo": "owner/name",
    "path": "path/to/file" ,
    "rule_id": "semgrep.rule.name",
    "severity": "WARNING|ERROR",
    "owasp": ["A03:2025 - ..."],
    "status": "present|fixed"
  }
}

🏗️ How the Data Was Generated

✔️ 1. Repo Selection

We selected GitHub repos from hacktoberfest_repos_2025.csv, filtering the top N repositories by stars, creation date, and topical relevance.

✔️ 2. Two Snapshots per Repo

• —Before: last commit ≤ 2025-10-01
• —After: first commit ≥ 2025-11-01

✔️ 3. Semgrep Scan (OWASP Top 10 Multilang)

Both snapshots were scanned using:

owasp-topten-multilang.yml

Findings include:

• —check_id
• —message & metadata
• —OWASP mappings
• —severity
• —file, path, and code ranges

✔️ 4. Per-Finding Code Extraction

Each finding includes:

• —The affected code range (with ±1 line context)
• —Git diff between snapshots
• —The finding's persistence/fix status

✔️ 5. SFT Example Construction

We generated two aligned datasets:

🟦 Repo-level

Natural-language summary of security posture changes:

• —Δ total findings
• —Δ OWASP categories
• —Δ rule counts
• —High-risk flags
• —Explanation sentences

🟩 Snippet-level

Instructional examples showing:

• —The insecure code
• —The fixed code (if applicable)
• —The diff
• —Explanation of why the original pattern was insecure

🎯 Intended Use Cases

• —Fine-tuning models to detect security vulnerabilities
• —Teaching LLMs how OWASP categories map to real code patterns
• —Training critics to judge code security (“fortification”)
• —Building automated code remediation agents
• —Benchmarking LLM-based static analysis

This dataset is optimized for:

• —TRL SFTTrainer
• —LoRA fine-tuning
• —Reward modeling / preference learning
• —Agent-based security critics (e.g., Durinn MCP)

⚠️ Limitations

• —Not all repos contain fixes; some only regress.
• —Semgrep rule coverage varies by language.
• —OWASP category extraction depends on rule metadata correctness.

📜 License

This dataset includes:

• —Small code excerpts from public GitHub repositories (fair use for research)
• —Derived annotations and summaries

Contact for DMCA/removal requests.

🙌 Acknowledgments

Special thanks to:

• —Hacktoberfest maintainers
• —Semgrep.io team
• —Durinn Research Group

📎 Citation

@dataset{durinn_hacktoberfest_retrospective,
  title={Durinn Hacktoberfest Retrospective},
  author={Marinelli, Ryan},
  year={2025},
  publisher={HuggingFace Datasets},
  howpublished={\url{https://huggingface.co/datasets/zrmarine/Durinn_Hacktoberfest_Retrospective}}
}