idkuk67/PromptShield_Arena_Openenv_2026

1---2title: PromptShield Arena3emoji: 🚀4colorFrom: indigo5colorTo: pink6sdk: docker7app_file: Dockerfile8pinned: false9---10# PromptShield Arena (OpenEnv)11[![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)12[![Hugging Face Space](https://img.shields.io/badge/HF%20Space-PromptShield_Arena_Openenv_2026-yellow)](https://huggingface.co/spaces/idkuk67/PromptShield_Arena_Openenv_2026)13 14## Problem Statement15Prompt injection is a real-world safety risk for AI systems. Agents must learn to detect unsafe prompts, explain why they are unsafe, and stay consistent under pressure. We need a reproducible OpenEnv environment that trains and evaluates this capability across difficulty levels.16 17## Solution (PromptShield Arena)18PromptShield Arena is an OpenEnv-compliant environment that generates infinite safe/unsafe prompts (easy → hard), scores agent decisions with shaped rewards, and provides feedback explaining unsafe cues. It exposes standard reset/step/state APIs, a baseline inference script, and a web UI for manual evaluation.19 20PromptShield Arena is a real-world OpenEnv environment for **prompt-injection detection**. It simulates a human task: screening user prompts for injection attempts and optionally providing safe rewrites.21## What This Does22- Simulates real-world prompt screening for injection attempts23- Gives shaped rewards for correct classification and helpful explanations24- Three difficulty levels with infinite, non-repeating prompts25 26## How To Play (UI)271. Click **Start Round**.282. Read the prompt and choose **Safe** or **Unsafe**.293. (Optional) Add a short explanation.304. Click **Submit** to get feedback and the next prompt.315. Switch levels anytime; the round resets automatically.32 33## Tasks (Easy → Medium → Hard)34- **easy**: obvious injection phrases35- **medium**: obfuscated or multi-step attempts36- **hard**: multi-turn or conflicting instruction attacks37 38Each task returns a score strictly between 0 and 1 (we clamp to 0.1–0.9) with partial credit for correct detection and mitigation.39 40## Game Mechanics (UI)41- Start a round to receive a prompt42- Decide Safe or Unsafe and (optionally) explain why43- Each wrong decision costs 1 life; Easy has infinite lives, Medium has 3, Hard has 544- A round-over screen appears; start a new round with full lives45- Score, streak, and accuracy update after every step46 47 48## Architecture49```mermaid50flowchart LR51  A[Prompt] --> B[Detector<br/>Policy + Heuristics]52  B --> C{Decision<br/>Safe / Unsafe}53  C --> D[Reward + Feedback]54  D --> E[Next Prompt]55```56 57## Examples58- `examples/sample_prompts.jsonl` shows sample prompts with expected labels.59- `examples/sample_output.txt` shows stdout format for a short run.60 61 62## Resource Constraints63Designed to run within **2 vCPU / 8 GB RAM** constraints.64 65## Roadmap66- Add PyTest coverage for environment logic.67- Add GitHub Actions CI for validation and linting.68- Add optional leaderboard mode to track highest-scoring agents globally.69- **Multilingual Prompts**: Expand the prompt generator to support safe/unsafe prompts in multiple languages to test multilingual robustness.70- **Dynamic Difficulty**: Automatically scale the challenge level based on the agent's current streak and accuracy.71- **Real-world Jailbreak Datasets**: Integrate community datasets (e.g., JailbreakChat) for real-world attack evaluations.72 73## Changelog74- **2026-04-11**: Added wrong-answers review panel and UI polish.75- **2026-04-10**: Added infinite prompt generation and feedback tags.76 77## Action Space78`PromptShieldAction`79- `decision`: "safe" or "unsafe"80- `explanation`: optional reasoning string81 82## Observation Space83`PromptShieldObservation`84- `prompt_id`, `prompt_text`, `task_level`85- `round_index`, `total_rounds`, `lives`, `streak`86- `total_score`, `average_score`, `attempts`, `correct_count`87- `done`, `reward`, `feedback`88 89## Environment Variables90See `.env.example`.91- `API_BASE_URL` (LLM endpoint)92- `MODEL_NAME` (LLM model id)93- `HF_TOKEN`94- `ENV_BASE_URL` (OpenEnv server, default `http://localhost:8000`)95 96## Local Setup971. Install server deps:98   ```bash99   pip install -r server/requirements.txt100   ```1012. Run the server:102   ```bash103   uvicorn server.app:app --host 0.0.0.0 --port 8000104   ```1053. Open the UI:106   ```107   http://localhost:8000108   ```1094. Run baseline inference:110   ```bash111   pip install -r requirements.txt112   python inference.py113   ```114 115## Validation116Run before submission:117```bash118openenv validate119```120 121## Files122- `openenv.yaml`: OpenEnv manifest123- `models.py`: typed action/observation/state124- `server/environment.py`: environment logic125- `server/app.py`: FastAPI app + UI126- `server/static/`: web UI127- `server/Dockerfile`: HF Spaces container128- `client.py`: OpenEnv EnvClient129- `inference.py`: baseline inference script130- `CONTRIBUTING.md`: contributor guide131- `CODE_OF_CONDUCT.md`: community standards132- `examples/`: sample prompts and stdout examples133- `logs/`: local run logs (empty in repo)134