logicBombExe/direct_prompt_injection_defense_data

Direct Prompt Injection Defense Dataset

Goal

This dataset is used to fine-tune models so they develop a natural defense against direct prompt injection attacks — without relying on external filters or guardrails.

Each example teaches the model two behaviors at once:

1. 1.Detect a prompt injection attempt in the user input.
2. 2.Respond correctly: reject malicious attempts, or answer safely when the user's intent is benign — and in both cases call the log_security_incident tool to record the incident.

How to Contribute

Data lives in data/train.jsonl — one conversation per line. Examples can be synthetic or real (observed in the wild); both follow the same structure. Follow this structure when adding examples:

Record fields

• —id: dpi-XXXX (next sequential number)
• —attack_category: short snake_case label of the attack pattern
• —messages: exactly 4 messages, in this order:

1. 1.user — the attack attempt (or benign input containing attack-like text)
2. 2.assistant — content: null, with a tool_calls entry calling log_security_incident
3. 3.tool — the tool's response: {"status": "logged", "incident_id": "INC-XXXX"}
4. 4.assistant — the final message to the user (refusal or safe answer)

Each message uses the fields: content, images, role, thinking, tool_calls (the tool message also has tool_call_id and name).

Tool call arguments

{
  "incident_type": "snake_case attack type",
  "severity": "low | medium | high",
  "user_input_excerpt": "short excerpt of the attack text",
  "action_taken": "rejected | answered_safely"
}

Severity and action rules

• —rejected + medium/high — the user is deliberately attacking. The final message refuses and does not comply with any part of the injected instruction.
• —answered_safely + low — the user's intent is benign, but attack-like text appears in the input (e.g. a quoted article). The final message answers the real question, ignores the embedded instruction, and briefly notes the detection.

See existing examples in data/train.jsonl for the exact tone and format.