scthornton/deepseek-coder-6.7b-securecode

DeepSeek Coder 6.7B SecureCode

<div align="center">

Security-specialized code model fine-tuned on the [SecureCode](https://huggingface.co/datasets/scthornton/securecode) dataset

Dataset | Paper (arXiv:2512.18542) | Model Collection | perfecXion.ai

</div>

What This Model Does

This model generates secure code when developers ask about building features. Instead of producing vulnerable implementations (like 45% of AI-generated code does), it:

• —Identifies the security risks in common coding patterns
• —Provides vulnerable and secure implementations side by side
• —Explains how attackers would exploit the vulnerability
• —Includes defense-in-depth guidance: logging, monitoring, SIEM integration, infrastructure hardening

The model was fine-tuned on 2,372 security training examples covering both traditional web security (OWASP Top 10 2021) and AI/ML security (OWASP LLM Top 10 2025).

Model Details

Strong code generation model with excellent fill-in-the-middle capabilities. Competitive with larger models on coding benchmarks.

Quick Start

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import torch

# Optional: 4-bit quantization for low-VRAM inference (training was bf16 LoRA)
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

base_model = AutoModelForCausalLM.from_pretrained(
    "deepseek-ai/deepseek-coder-6.7b-instruct",
    quantization_config=bnb_config,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("scthornton/deepseek-coder-6.7b-securecode")
model = PeftModel.from_pretrained(base_model, "scthornton/deepseek-coder-6.7b-securecode")

# Ask a security-relevant coding question
messages = [
    {"role": "user", "content": "How do I implement JWT authentication with refresh tokens in Python?"}
]

inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
outputs = model.generate(inputs, max_new_tokens=2048, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Dataset

Trained on the full [SecureCode](https://huggingface.co/datasets/scthornton/securecode) unified dataset:

• —2,372 total examples (1,625 web security + 747 AI/ML security)
• —20 vulnerability categories across OWASP Top 10 2021 and OWASP LLM Top 10 2025
• —12+ programming languages and 49+ frameworks
• —4-turn conversational structure: feature request, vulnerable/secure implementations, advanced probing, operational guidance
• —100% incident grounding: every example tied to real CVEs, vendor advisories, or published attack research

Hyperparameters

Notes: Compact LoRA targeting attention layers only (4 modules). Extended 4096-token context.

Security Coverage

Web Security (1,625 examples)

OWASP Top 10 2021: Broken Access Control, Cryptographic Failures, Injection, Insecure Design, Security Misconfiguration, Vulnerable Components, Authentication Failures, Software Integrity Failures, Logging/Monitoring Failures, SSRF.

Languages: Python, JavaScript, Java, Go, PHP, C#, TypeScript, Ruby, Rust, Kotlin, YAML.

AI/ML Security (747 examples)

OWASP LLM Top 10 2025: Prompt Injection, Sensitive Information Disclosure, Supply Chain Vulnerabilities, Data/Model Poisoning, Improper Output Handling, Excessive Agency, System Prompt Leakage, Vector/Embedding Weaknesses, Misinformation, Unbounded Consumption.

Frameworks: LangChain, OpenAI, Anthropic, HuggingFace, LlamaIndex, ChromaDB, Pinecone, FastAPI, Flask, vLLM, CrewAI, and 30+ more.

SecureCode Model Collection

This model is part of the SecureCode collection of 9 security-specialized models:

Choose based on your deployment constraints: 3B for edge/mobile, 7B for general use, 13B-15B for deeper reasoning, 20B for maximum capability.

SecureCode Dataset Family

Intended Use

Use this model for:

• —Training AI coding assistants to write secure code
• —Security education and training
• —Vulnerability research and secure code review
• —Building security-aware development tools

Do not use this model for:

• —Offensive exploitation or automated attack generation
• —Circumventing security controls
• —Any activity that violates the base model's license

Changelog

• —2026-07 (v2, current): Retrained on the audited SecureCode release (2,372 examples: 1,625 web + 747 AI/ML) using bf16 LoRA on an NVIDIA DGX Spark GB10 (Blackwell). Same LoRA shape and hyperparameters as v1, except: no quantization during training, optimizer adamwtorchfused instead of pagedadamw8bit, PyTorch SDPA attention, dynamic per-batch padding. A new family member, gemma-4-e4b-securecode, was added in this refresh.
• —v1 (2026-02): QLoRA 4-bit NF4 on NVIDIA A100 40GB, trained on the pre-audit 2,185-example dataset.

Citation

@misc{thornton2026securecode,
  title={SecureCode: A Production-Grade Multi-Turn Dataset for Training Security-Aware Code Generation Models},
  author={Thornton, Scott},
  year={2026},
  publisher={perfecXion.ai},
  url={https://huggingface.co/datasets/scthornton/securecode},
  note={arXiv:2512.18542}
}

Links

• —Dataset: scthornton/securecode
• —Research Paper: arXiv:2512.18542
• —Model Collection: huggingface.co/collections/scthornton/securecode
• —Author: perfecXion.ai

License

This model is released under the other license (inherited from the base model). The training dataset (SecureCode) is licensed under CC BY-NC-SA 4.0.