aneetk09/prompt-injection-env

Prompt Injection Detection Environment

An OpenEnv RL environment for training and evaluating AI agents on LLM prompt injection detection — the #1 security threat for deployed AI applications.

What It Does

The agent reads conversation contexts and retrieved document content and must:

1. 1.Detect whether a prompt injection attack is present
2. 2.Classify the attack vector (direct / indirect / multi-vector)
3. 3.Assess severity (low / medium / high / critical)
4. 4.Localize the injected content

Tasks

Action Space

JSON object with fields: injectiondetected (bool), attackvector (string), severity (string), location (int or string or list), explanation (string)

Observation Space

JSON object with: scenario (messages array + optional retrievedcontext), scoreso_far (float), done (bool), step (int)

Reward Function

Weighted composite score (0.0 to 1.0):

• —Detection accuracy: 40%
• —Attack vector classification: 25%
• —Severity classification: 20% (partial credit for off-by-one)
• —Location identification: 15%

Setup

pip install -r requirements.txt uvicorn main:app --host 0.0.0.0 --port 7860

Docker

docker build -t prompt-injection-env . docker run -p 7860:7860 prompt-injection-env

Baseline Inference

export HFTOKEN=yourtoken export APIBASEURL=https://router.huggingface.co/v1 export MODELNAME=Qwen/Qwen2.5-72B-Instruct export ENVBASE_URL=http://localhost:7860 python inference.py

Baseline Scores