SidhaGarg/Cloud-DevOps-RLEnv

1---2title: Cloud-DevOps-RLEnv3emoji: ☁️4colorFrom: blue5colorTo: gray6sdk: docker7app_port: 80008tags:9- openenv10---11 12# Cloud DevOps RLEnv13 14Cloud DevOps RLEnv is an OpenEnv-compatible cloud incident-response benchmark designed for agentic SRE and DevOps workflows.15 16This environment rewards correct diagnosis and safe remediation, not blind action execution. It is deterministic, reproducible, and optimized for hackathon evaluation.17 18## Judge-Aligned Snapshot19 20| Parameter | Weight | How this environment addresses it |21| --- | --- | --- |22| Real-world utility | 30% | Models practical SRE outage response loops: telemetry triage, dependency mapping, and safe remediation. |23| Task & grader quality | 25% | Three deterministic tasks with explicit objectives, strict success gates, and reproducible scoring behavior. |24| Environment design | 20% | Typed action/observation/state models, clean reset semantics, shaped rewards, action-cost efficiency pressure, clear boundaries. |25| Code quality & spec compliance | 15% | OpenEnv-compliant project layout, Dockerized runtime, strict inference output contract, validation scripts. |26| Creativity & novelty | 10% | Multi-hop metadata dependency, cascading failures, high-decoy search space, and safety-aware penalties. |27 28## Why This Environment29 30Real incidents are multi-step and noisy. Good agents must:31- gather context before changing systems32- identify root cause from logs and topology33- apply minimal, correct fixes34- verify resolution35 36Cloud DevOps RLEnv simulates that behavior with realistic failure patterns, decoy resources, shaped rewards, and anti-shortcut guardrails.37 38## Why It's Hard39 40This benchmark is intentionally designed to resist brute-force policies and reward disciplined SRE reasoning:41 42- Needle-in-a-haystack discovery: 20+ decoy compute nodes and 20+ decoy security groups increase search complexity.43- Ambiguous telemetry: noisy, raw operational logs surface symptoms (including IP-only clues) rather than direct root-cause labels.44- Action-penalty heuristics: every action has a small negative cost, so efficient remediation beats command spamming.45- Multi-hop dependency resolution: agents must map IP addresses to resource IDs via metadata lookup before applying fixes.46- System drift under pressure: in hard mode, delayed remediation triggers cascading failures that worsen observability and reward dynamics.47 48## Environment Scope49 50- Domain: Cloud SRE / DevOps incident response51- Difficulty tiers: easy, medium, hard52- Max environment steps per episode: 2053- Runtime health states: CRITICAL, DEGRADED, HEALTHY54- Decoy resources: 20 backend instances + 20 backend security groups55 56## OpenEnv Compliance57 58Core files:59- openenv.yaml60- env.py61- models.py62- inference.py63- server/app.py64- server/cloud_devops_env_environment.py65 66Validator command:67 68```bash69..\\.venv\\Scripts\\openenv validate70```71 72## Action Space73 74Model: CloudAction75 76| Field | Type | Required | Description |77| --- | --- | --- | --- |78| command | enum | yes | One of: list_resources, describe_resource, view_logs, query_metadata, update_security_group, restart_service, submit_solution |79| resource_id | string | conditional | Required for most actions except list_resources and query_metadata |80| parameters | object | conditional | Used by mutating actions (for example, security-group updates) |81 82Action semantics:83- list_resources: Enumerates available resources including decoys.84- describe_resource: Returns structured details for one resource.85- view_logs: Returns logs for one resource.86- query_metadata: Resolves infrastructure metadata (for example, IP address to resource ID).87- update_security_group: Appends a rule (requires parameters.port and parameters.action where action is allow/deny).88- restart_service: Restarts one instance/service by ID.89- submit_solution: Declares the episode solved (or not solved).90 91## Observation And State Space92 93Observation model: CloudObservation94 95| Field | Description |96| --- | --- |97| output | Main command output |98| error | Error string for failed commands |99| system_health_status | CRITICAL, DEGRADED, HEALTHY |100| done | Episode terminal flag |101| reward | Step reward |102| metadata | Diagnostics such as task, step_count, resolved, achievements |103 104Hidden state model: CloudState105 106| Field | Description |107| --- | --- |108| task_difficulty | easy, medium, hard |109| resources | Full resource graph including logs/rules |110| step_count | Current step counter |111| is_resolved | Whether root cause has been fixed |112 113## Reward Design114 115Reward shaping is sparse-but-guided:116- discovery rewards for correct investigative steps117- larger terminal rewards for correct remediation118- penalties for unsafe or premature operations119- fixed action cost per step (efficiency pressure)120- timeout terminal condition after max steps121 122Per-step reward is clipped to [-1.0, 1.0].123Inference task score is adjusted to remain strictly within (0.0, 1.0) for Phase-2 validator compatibility.124 125## Detailed Task Playbooks126 127### Easy Task128 129Incident:130- Web traffic blocked by security group.131 132Objective:133- Open port 80 on sg-web.134 135Typical successful sequence:1361. list_resources1372. describe_resource(sg-web) for context (+0.2)1383. update_security_group(sg-web, port=80, action=allow) (+0.8, done)139 140Expected score:141- ~0.97 for full playbook with efficient triage142- ~0.79 if agent skips the optional read step143 144### Medium Task145 146Incident:147- API cannot reach DB due to blocked port 5432.148 149Objective:150- Confirm root cause from logs, then open port 5432 on sg-db.151 152Typical successful sequence:1531. list_resources1542. view_logs(i-api) to identify DB timeout (+0.2)1553. query_metadata(ip_address=10.0.4.5) to resolve DB target (+0.2)1564. update_security_group(sg-db, port=5432, action=allow) (+0.6, done if logs and metadata lookup were completed)157 158Guardrail:159- Applying the SG change before log triage + metadata lookup gives a penalty (-0.1) and does not close the incident.160 161Expected score:162- ~0.97 with full investigative path (logs -> metadata lookup -> remediation)163- below ~0.90 when metadata dependency is skipped164 165## Determinism And Grader Transparency166 167- Deterministic reset and transitions: no randomization is used in task generation or transition logic.168- Transparent grading signals: observation metadata includes achievements, resolution status, termination reason, and reward breakdown events.169- Reproducibility helper:170 171```bash172..\\.venv\\Scripts\\python scripts/reproducibility_check.py173```174 175### Hard Task176 177Incident:178- Checkout path degraded due to upstream timeout to an IP-only target that must be resolved first.179 180Objective:181- Trace LB errors to the correct target, resolve resource identity via metadata, and restart i-web2 only after diagnosis.182 183Typical successful sequence:1841. list_resources1852. view_logs(lb-main) to identify failing upstream IP (+0.2)1863. query_metadata(ip_address=<failing_ip>) to resolve target ID (+0.2)1874. describe_resource(i-web2) or view_logs(i-web2) (+0.2)1885. restart_service(i-web2) (+0.8, done when all investigation achievements exist)189 190Guardrails:191- Restarting i-web2 before investigation: penalty (-0.1), no resolution.192- Restarting healthy i-web1: penalty (-0.2).193- Premature submit_solution in hard mode: penalty (-0.1), episode continues.194- If unresolved after step 8 in hard mode, lb-external also fails (cascading failure), increasing pressure and noise.195 196Expected score:197- near 1.0 after score clamping for strong trajectories (can exceed 1.0 raw before clamp)198 199## API Endpoints200 201Core runtime:202- GET /health203- POST /reset204- POST /step205- GET /state206- GET /schema207- WS /ws208 209Web UI runtime:210- GET /web211- POST /web/reset212- POST /web/step213- GET /web/state214- GET /web/metadata215 216## Inference Contract217 218inference.py requirements:219- uses OpenAI client220- reads API_BASE_URL, MODEL_NAME, HF_TOKEN221- emits strict logs: [START], [STEP], [END]222 223Current defaults in code:224- MODEL_NAME default: google/gemma-4-26B-A4B-it225- MAX_STEPS (in inference loop): 15226- success flag is derived from environment resolution state227 228## Baselines229 230### Deterministic task baselines231 232| Task | Typical baseline score |233| --- | --- |234| easy | 0.78 |235| medium | 0.96 |236| hard | 0.999 |237 238### LLM policy comparison239 240| Model | Easy | Medium | Summary |241| --- | --- | --- | --- |242| gemma-3-27b-it | 0.2 | 0.2 | Underperformed on this environment |243| gemma-4-31b-it | 1.0 | 1.0 | Perfect on both easy and medium |244 245## Local Setup And Validation246 247From repository root:248 249```bash250# Structure + manifest validation251..\\.venv\\Scripts\\openenv validate252 253# Determinism/reproducibility smoke test254..\\.venv\\Scripts\\python scripts/reproducibility_check.py255 256# Submission-oriented local checks (without live inference)257bash scripts/pre_submit_validate.sh --skip-inference258 259# Build local image260docker build -t cloud-devops-env:phase1 -f Dockerfile .261```262 263Optional local server:264 265```bash266uvicorn server.app:app --host 0.0.0.0 --port 8000267```268 269## Hugging Face Space Deployment270 2711. Keep this front matter block intact (includes mandatory openenv tag).2722. Push to Space (Docker SDK).2733. Configure secrets/variables:274   - HF_TOKEN275   - API_BASE_URL (for example https://router.huggingface.co/v1)276   - MODEL_NAME2774. Wait for build completion.2785. Verify:279   - GET /health returns 200280   - POST /reset returns 200281 282Reference:283- https://huggingface.co/docs/hub/spaces-config-reference284