brandonzrw/Qwen2.5-Coder-7B-Instruct-Top25-CWE-LoRA-FineTuned

Model Card for Qwen2.5-Coder-7B-Instruct-Top25-CWE-LoRA-FineTuned

This model is a fine-tuned version of Qwen/Qwen2.5-Coder-7B-Instruct. It has been trained using TRL.

Intended use

Given a source code function, this model generates a structured vulnerability report as JSON, restricted to 25 target CWE types:

{
  "cwe_type": "CWE-787",
  "explanation": "A single technical sentence explaining the flaw.",
  "exploit_path": "A theoretical attack scenario describing how this is triggered.",
  "impact": "e.g. Remote Code Execution",
  "mitigation": "A specific code-level fix.",
  "sink_or_region": "The exact snippet of code causing the vulnerability."
}

Usage

This adapter was trained and evaluated with vLLM using schema-constrained decoding (StructuredOutputsParams), not plain transformers generation — the exact system prompt, sampling settings, and JSON-schema enforcement it expects are already implemented in `inference_scripts/` in the project repo. Refer to zero_shot_inference.py or few_shot_inference.py there for a working, tested example of how to load and query this adapter correctly.

Training data

Fine-tuned on a mix of two vulnerability corpora, filtered down to 25 target CWE types and annotated with structured vulnerability reports (explanation, exploit path, impact, mitigation, sink/region) by a larger teacher LLM:

• —[TitanVul](https://huggingface.co/datasets/yikun-li/TitanVul) — the raw corpus is not redistributed here due to its size (200MB+); the filtered/annotated derivatives used for training are included in the project repo: `Filtered_Datasets/titanvul_filtered.jsonl`, `Annotated_Datasets/titanvul_qwen_annotated_dataset.jsonl`.
• —LLM-Vul — raw and annotated files included directly: `Raw_Datasets/llm_vul_dataset.jsonl`, `Annotated_Datasets/llm_vul_qwen_annotated_dataset_cleaned.jsonl`.

This adapter was trained specifically on the resulting train split: `Annotated_Datasets/titanvul_train_test_split/train.jsonl` (7,360 rows).

Training procedure

This model was trained with SFT, using QLoRA — a 4-bit NF4-quantized base model (bitsandbytes, double quantization) with LoRA adapters trained in bf16 — via TRL's SFTTrainer.

LoRA config: rank 16, alpha 32, dropout 0.05, no bias, applied to all attention and MLP projections (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj).

Training hyperparameters:

• —Effective batch size 8 (per-device batch size 1 × gradient accumulation 8), across 2 GPUs
• —Learning rate 2e-5, cosine schedule, 3% warmup, max grad norm 0.3
• —3 epochs, max sequence length 4096, gradient checkpointing
• —Early stopping (patience 3) on eval loss; evaluated and checkpointed every 100 steps, best checkpoint kept
• —90/10 train/validation split, stratified by CWE type

Run stats: 7,360 training examples, 1,242 total optimizer steps, ~9h43m wall-clock time on the run captured in `lora_training_log.txt`.

Framework versions

• —PEFT 0.17.1
• —TRL: 0.26.2
• —Transformers: 4.57.1
• —Pytorch: 2.8.0
• —Datasets: 4.4.2
• —Tokenizers: 0.22.1

Citations

Cite TRL as:

@misc{vonwerra2022trl,
	title        = {{TRL: Transformer Reinforcement Learning}},
	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
	year         = 2020,
	journal      = {GitHub repository},
	publisher    = {GitHub},
	howpublished = {\url{https://github.com/huggingface/trl}}
}