khushmagrawal/devsecops_env

1---2title: DevSecOps Gatekeeper Environment3emoji: 🚀4colorFrom: blue5colorTo: purple6sdk: docker7pinned: false8app_port: 80009base_path: /web10tags:11  - openenv12  - security13  - reinforcement-learning14  - devsecops15---16 17# DevSecOps Gatekeeper Environment18 19An advanced OpenEnv RL environment where AI agents learn to make high-stakes security decisions on incoming Pull Requests. The agent uses simulated tools to analyze code changes, run CI/CD pipelines, patch code, scan vulnerabilities, and ultimately approve or block PRs.20 21## Overview22 23This environment presents three progressively harder scenarios:24 25### Task 1: Docs-Only PR26**Complexity**: Easy27 28A PR that ONLY changes documentation and comments. The agent should recognize this and approve without unnecessary testing.29 30- **Optimal path**: inspect_diff → make_decision(MERGE)31- **Optimal reward**: ~0.99932- **Key skill**: Recognition - identify zero-risk changes33 34### Task 2: Silent API Rename35**Complexity**: Medium36 37A package dependency bump (httpx 0.23.0 → 0.28.0) has breaking API changes. The agent must:381. Detect the breaking change392. Run CI to see the failure403. Patch the code to use the new API414. Verify the fix with CI425. Approve the PR43 44- **Optimal path**: inspect_diff → run_ci (fail) → query_registry → patch_code → run_ci (pass) → make_decision(MERGE)45- **Optimal reward**: ~0.99946- **Key skill**: Remediation - fix breaking dependency changes47 48### Task 3: Poisoned Package49**Complexity**: Hard50 51A package (cryptoutils 2.1.5) contains malware in its setup.py that exfiltrates system information. The agent must:521. Detect malicious code patterns532. Check suspicious package metadata (new maintainer, ownership transfer)543. Block the malicious package554. CRUCIALLY: Avoid running CI (which would execute the malware)56 57- **Optimal path**: inspect_diff → query_registry → make_decision(BLOCK)58- **Optimal reward**: ~0.99959- **Key skill**: Security - detect supply chain attacks60 61## Installation62 63```bash64# Install the environment package65pip install -e .66 67# Or with development dependencies68pip install -e ".[dev]"69```70 71## Quick Start72 73### Using the Python Client74 75```python76from devsecops_env import DevsecopsEnv, DevsecopsAction77 78# Connect to locally running server79with DevsecopsEnv(base_url="http://localhost:8000") as client:80    # Reset to start a new episode81    result = client.reset()82    print(f"Task: {result.observation.task_id}")83    84    # Inspect PR changes85    action = DevsecopsAction(tool_name="inspect_diff")86    result = client.step(action)87    print(f"Diff: {result.observation.last_tool_output[:200]}...")88    89    # Make decision90    action = DevsecopsAction(91        tool_name="make_decision",92        verdict="MERGE",93        justification="Docs only, no functional changes"94    )95    result = client.step(action)96    print(f"Done: {result.done}, Reward: {result.observation.episode_reward}")97```98 99### Starting the Server Locally100 101```bash102# Install dependencies103uv sync104 105# Start server106cd devsecops_env && uvicorn server.app:app --reload --port 8000107```108 109The server will be available at `http://localhost:8000` with:110- REST API endpoints for reset/step/state111- WebSocket support for persistent sessions112- Gradio web interface at `/web`113 114### Running Tests115 116```bash117# Run comprehensive test suite118python test_env.py119 120# Or with pytest121pytest test_env.py -v122```123 124Tests validate:125- Scenario loading and integrity126- Tool dispatcher behavior127- State transitions and tracking128- Reward calculations129- End-to-end episode flows130 131## Environment API132 133### Action Schema134 135`DevsecopsAction` contains:136- `tool_name` (required): One of ["inspect_diff", "run_ci", "patch_code", "query_package_registry", "search_vuln_db", "make_decision"]137- Tool-specific parameters (all optional): `pr_id`, `scope`, `pkg`, `version`, `file`, `old_code`, `new_code`, `verdict`, `justification`138 139Example actions:140 141```python142# Inspect PR changes143inspect_action = DevsecopsAction(tool_name="inspect_diff", pr_id="pr_001")144 145# Run CI with specific scope146ci_action = DevsecopsAction(tool_name="run_ci", scope="unit_only")147 148# Patch code (Task 2)149patch_action = DevsecopsAction(150    tool_name="patch_code",151    file="src/api_client.py",152    old_code="await client.send(request)",153    new_code="await client.request('GET', url)"154)155 156# Query package registry157query_action = DevsecopsAction(158    tool_name="query_package_registry",159    pkg="httpx",160    version="0.28.0"161)162 163# Search vulnerability databases164vuln_action = DevsecopsAction(165    tool_name="search_vuln_db",166    pkg="cryptoutils",167    version="2.1.5"168)169 170# Make final decision171decision_action = DevsecopsAction(172    tool_name="make_decision",173    verdict="MERGE",  # or "REQUEST_CHANGES" or "BLOCK"174    justification="Security checks passed"175)176```177 178### Observation Schema179 180`DevsecopsObservation` contains:181- `task_id`: Current task being solved182- `pr`: Pull request metadata183- `repo_context`: Repository information184- `budget`: Remaining CI runs and step limit185- `pipeline_history`: All tool calls made so far186- `last_tool_output`: Text output from most recent tool187- `done`: Episode completion flag188- `reward`: Reward from last step189- `episode_reward`: Cumulative reward190- `step_count`: Total steps taken191- `internal_state`: Task-specific mutable state (code_patched, etc)192 193## Tools Explained194 195### inspect_diff196Analyzes the PR diff to understand what's changing. Returns:197- Summary of files changed198- Analysis (docs-only? code changes? breaking changes?)199- Red flags or warnings200 201### run_ci202Runs the CI/CD pipeline. Results depend on:203- Task ID204- Current state (e.g., whether code was patched in Task 2)205- Scope: "unit_only" or "full"206 207Cost: 1 CI run (budget is limited)208 209### patch_code210Attempts to patch code. For Task 2, validates that the patch:211- Replaces deprecated API calls212- Makes semantic sense213 214Success marks code as patched in internal state, affecting subsequent CI runs.215 216### query_package_registry217Looks up package metadata from PyPI/registry:218- Maintainer information (age, prior releases)219- Download statistics220- Ownership transfer history221- Notes on suspicious patterns222 223### search_vuln_db224Searches CVE and OSV vulnerability databases:225- Known CVEs226- Suspicious code patterns detected227- Notes on package age (very new packages have no history)228 229### make_decision230Terminal action that ends the episode. Sets verdict ("MERGE", "REQUEST_CHANGES", or "BLOCK") and triggers reward calculation.231 232## Reward Structure233 234All rewards are normalized to the range `(0, 1)`.235 236### Task 1 (Docs-Only)237- Correct verdict (MERGE): High reward (~0.999)238- Incorrect verdict (BLOCK): Very low reward (~0.0001)239- Penalty: Slight reduction for each unnecessary CI run.240 241### Task 2 (Silent API Rename)242- Correct verdict with patch: Optimal reward (~0.999)243- Verdict without patch: Negative sentiment reflected in low reward.244- Penalty: Reduction for excessive CI runs beyond optimal (2).245 246### Task 3 (Poisoned Package)247- Correct verdict (BLOCK): High reward (~0.999)248- Incorrect verdict (MERGE): Catastrophic failure (~0.0001)249- Penalty: Significant reduction for each CI run (as CI executes malware).250 251## State Management252 253The environment uses **per-episode mutable state** to enable:254- **Task 2**: Tracking whether code has been patched (affects CI results)255- **Task 3**: Stateless (each tool call returns deterministic output)256 257Each `reset()` creates a fresh, isolated episode state.258 259## Docker Deployment260 261Build the Docker image:262 263```bash264docker build -t devsecops_env:latest server/265```266 267Run locally:268 269```bash270docker run -p 8000:8000 devsecops_env:latest271```272 273## Deploying to Hugging Face Spaces274 275```bash276huggingface-cli login277openenv push278```279 280Pushes the environment to Hugging Face Spaces with automatic Docker building and Gradio web interface.281 282## File Structure283 284```285devsecops_env/286├── __init__.py                    # Package exports287├── models.py                      # Pydantic schemas (Action, Observation)288├── client.py                      # HTTP/WebSocket client289├── test_env.py                    # Comprehensive test suite290├── openenv.yaml                   # OpenEnv manifest291├── pyproject.toml                 # Package configuration292├── README.md                      # This file293└── server/294    ├── __init__.py295    ├── app.py                     # FastAPI application296    ├── devsecops_env_environment.py  # Core environment logic297    ├── mock_tools.py              # Tool implementations298    ├── graders.py                 # Reward calculation299    ├── requirements.txt300    ├── Dockerfile301    └── scenarios/302        ├── __init__.py            # Registry and loader303        ├── task1.py               # Docs-only scenario304        ├── task2.py               # Silent API rename scenario305        └── task3.py               # Poisoned package scenario306```307 308## References309 310- [OpenEnv Specification](https://github.com/huggingface/openenv-course)311- [Meta OpenEnv Repository](https://github.com/meta-pytorch/OpenEnv)312- [Gymnasium API](https://gymnasium.farama.org/)313 314openenv push --base-image ghcr.io/meta-pytorch/openenv-base:latest315 316# Push as a private space317openenv push --private318 319# Combine options320openenv push --repo-id my-org/my-env --base-image custom-base:latest --private321```322 323After deployment, your space will be available at:324`https://huggingface.co/spaces/<repo-id>`325 326The deployed space includes:327- **Web Interface** at `/web` - Interactive UI for exploring the environment328- **API Documentation** at `/docs` - Full OpenAPI/Swagger interface329- **Health Check** at `/health` - Container health monitoring330- **WebSocket** at `/ws` - Persistent session endpoint for low-latency interactions331 332## Development & Testing333 334### Direct Environment Testing335 336Test the environment logic directly without starting the HTTP server:337 338```bash339python test_env.py340```341 342### Starting the Server Locally343 344```bash345uvicorn server.app:app --reload346```347