darwhite08/OpenEnv-SecOps

<div align="center">

<!-- HERO IMAGE --> <div align="center"> <img src="/OpenEnv-SecOps/media/header_image.png" alt="OpenEnv-SecOps Autonomous SOC" width="200" /> <p><b>OpenEnv-SecOps — Autonomous SOC Analyst Simulation</b></p> </div>

🛡️ OpenEnv SecOps

A Long-Horizon, Partially Observable RL Environment for Training Autonomous SOC Analysts

Theme #2 — (Super) Long-Horizon Planning & Instruction Following with elements of #3.1 Professional Tasks. Built on OpenEnv. Trains LLMs to decompose multi-step incident-response goals, track adversary state across noisy logs, recover from premature actions, and neutralize threats under irreversible decisions.

    

</div>

🔗 All submission links

⚠️ For judges: the URL above (HF Space) is the canonical environment endpoint. Pull from that.

📰 TL;DR — what we found. A 72B LLM facing a real-looking SOC console doesn't panic, it just keeps searching. Our APT-Winnti baseline came in at 10% success because the agent kept chasing lateral-movement alerts past the budget threshold. We added a 3-search hard cap, a hallucination guard that rejects invented user IDs / IPs, a smart-Done APT-decay path, and — embarrassingly — fixed a five-line env serialization bug that meant Done never actually terminated the run. Net result on a 60-episode sweep: 47% → 67% total success; APT-Winnti score `0.169` → `0.613` (3.6×); hard success 10% → 35%, no fine-tuning required. Full write-up in `blog.md`.

🎯 The capability gap we're targeting

Can an LLM, dropped into a noisy SIEM with 500 log lines and a single active breach, plan and execute a multi-tool remediation playbook under irreversible decisions — and recover when its early guesses are wrong?

Current LLMs fail this task out of the box. They:

1. 1.Hallucinate IPs that don't appear in logs.
2. 2.Burn budget repeating the same SearchSIEM query because they can't track what they've already done.
3. 3.Skip mandatory sequencing (Isolate → Reset → KillProcess → BlockPort → Done).
4. 4.Refuse to commit to destructive actions like Isolate_IP even when confident.

This environment measurably trains those failure modes out with a dense, composable reward signal that rewards correct sequencing and penalises noise/hallucination.

🏛️ Why this fits Theme #2

• —Long horizon, sparse intermediate reward. Up to 10 sequential tool calls; success rewarded only after the full Isolate → Identify → Remediate → Verify chain.
• —Partial observability. The agent sees logs incrementally — the breach IP, compromised user, and malicious PID are not in the initial alert; they must be discovered via SearchSIEM.
• —State that exceeds context. A persistent memory_palace field in the observation lets the agent stash discovered intel across turns without bloating the prompt.
• —Irreversible decisions. Isolate_IP on a Payment-Gateway is a permanent revenue loss; KillProcess on the wrong PID is wasted budget. The agent must reason under permanence.
• —Recovery from early mistakes. Wrong IP isolation is recoverable via further searches; the reward function distinguishes "wrong but recoverable" from "catastrophic."

<div align="center">

<!-- ARCHITECTURE DIAGRAM IMAGE --> <img src="https://placehold.co/1200x520/0b0b0c/22d3ee?text=Architecture%3A+Browser+%E2%87%84+FastAPI+%E2%87%84+Agent+%E2%87%84+SecopsEnv&font=montserrat" alt="OpenEnv-SecOps three-layer architecture: browser ↔ FastAPI sidecar ↔ secops_agent ↔ env" width="900" />

</div>

🏗️ Architecture

Three loosely-coupled layers behind a single Hugging Face Space.

┌───────────────────────────────────────────────────────────────────────────────┐
│  Browser  (live war room — Next.js 16 + React 19)                             │
│   useSecopsEvents()  ── EventSource ──┐                                       │
│   TriageQueue ── POST /api/actions/:id/{approve,reject} ──┐                   │
└────────────────────────────────────────┼──────────────────┼───────────────────┘
                                         │ same-origin SSE  │ REST + Bearer
                                         ▼                  ▼
┌───────────────────────────────────────────────────────────────────────────────┐
│  FastAPI sidecar  (port 7860 = HF Space port)                                 │
│   /api/events    SSE  ◄── EventBus (asyncio pub/sub + replay buffer)          │
│   /api/runs      POST/GET           /api/runs/:id/cancel                      │
│   /api/actions/:id/{approve,reject} ◄── HITL AwaitableGate                    │
│   /api/health    /api/metrics (Prometheus, opt-in)                            │
│   /                  ◄── StaticFiles (Next.js export, mounted at root)        │
└──────────────────────────┬────────────────────────────────────────────────────┘
                           │ publishes typed events
                           ▼
┌───────────────────────────────────────────────────────────────────────────────┐
│  secops_agent package                                                         │
│   agent.loop ─► env.reset / env.step (SecopsEnv — OpenEnv compliant)          │
│              ─► LLM client (HF Router → Qwen 2.5-72B / Groq / Ollama …)       │
│              ─► JSON parser  (balanced-brace, <think>-strip, multi-blob)      │
│              ─► action validator + corrective re-prompt                       │
│              ─► observation pruning (drop noise rows)                         │
│              ─► HITL gate for destructive low-confidence actions              │
│              ─► smart-Done heuristic                                          │
│              ─► token budget tracker, retry + circuit breaker                 │
└───────────────────────────────────────────────────────────────────────────────┘

🚀 Quick start

A. Run the live Hugging Face Space (one click)

1. 1.Open <https://huggingface.co/spaces/darwhite08/OpenEnv-SecOps>
2. 2.In Space Settings → Secrets, set HF_TOKEN.
3. 3.Click Start in the war-room UI. Watch the agent reason and act.

B. Local dev

git clone https://github.com/darwhite08/OpenEnv-SecOps.git
cd OpenEnv-SecOps
cp .env.example .env                     # then edit HF_TOKEN
uv sync                                  # Python 3.10+

docker build -f Dockerfile.env-only -t openenv-secops:latest .
uv run python inference.py serve --port 8765   # backend + UI

# In another terminal:
cd ui && npm install && SECOPS_API_URL=http://localhost:8765 npm run dev
# open http://localhost:3000

C. Headless single-task evaluation

uv run python inference.py run --task hard
# [START] task=hard env=SecOps-Env model=Qwen/Qwen2.5-72B-Instruct:novita
# [STEP] step=1 action=SearchSIEM(target=None) confidence=0.90 reward=-0.06 done=false error=null
# [STEP] step=2 action=Isolate_IP(target=10.0.0.5) confidence=0.95 reward=-0.10 done=false error=null
# [STEP] step=3 action=ResetUser(target=EMP-405)  confidence=0.95 reward=-0.05 done=true error=null
# [END] success=true steps=3 score=0.83 rewards=-0.06,-0.10,-0.05

⚔️ Action space

The agent picks one tool per turn. Every tool consumes budget; failing to neutralize the threat bleeds further per-tick.

🎁 Reward model (composable, dense, hard-to-game)

reward_t  =  budget_score_t  −  budget_score_{t−1}

budget_score_t  =  budget_score_{t−1}
                 −  per_action_cost(tool)
                 −  0.05 × active_infections_t
                 −  0.10 × {1 if Isolate_IP on Payment-Gateway else 0}
                 +  0.05 × {1 if correct sequence on this turn else 0}

Three properties make this hard to game:

1. 1.Dense intermediate signal. Reward updates every step, not only at termination — feeds RL gradient directly.
2. 2.Per-tick infection penalty. Stalling (e.g. SearchSIEM-spamming) bleeds budget while infections remain active. An agent that "surfs" on cheap actions still loses.
3. 3.Sequence bonus only on correct ordering. Isolate_IP → SearchSIEM(IP) → ResetUser(EMP) → KillProcess(pid) → Done triggers the bonus; out-of-order actions don't.

Success threshold: `budget_score ≥ 0.80` at episode end.

<div align="center">

<img src="docs/baseline_curve.png" alt="Baseline reward curve — Qwen-2.5-72B across 5 episodes per task" width="900" />

</div>

📈 Training results

📓 Training notebook: `notebooks/train_secops_agent.ipynb` — runs Unsloth + HF TRL against this OpenEnv environment in ~30 minutes on a free T4. 🛠 Training playground: `training/` — five-script pipeline (collect → prepare → train → evaluate → push) with YAML hyperparameters. 🚀 Trained inference Space: `darwhite08/QwenFineTuneModel` — OpenAI-compatible endpoint serving the LoRA-fine-tuned Qwen-2.5-1.5B.

Real measured baseline (Qwen-2.5-72B via HF Router :novita, 10 episodes per task)

The three scenarios escalate from a one-host phishing through to a multi-host APT:

Average score across all scenarios: 0.602. Total successes: 14/30 (47%).

(docs/baseline_results.json has the per-episode raw data; docs/baseline_table.md regenerates this table; scripts/eval_baseline.py --episodes N re-runs the sweep.)

Reward curves — what the budget actually does over a run

<div align="center"> <img src="docs/reward_curves.png" alt="Per-step cumulative reward across baseline runs, by scenario" width="950" /> </div>

Each gray line is one episode; the green line is the mean across the 10 episodes for that scenario. The dashed yellow line is the 0.80 success threshold.

Three things are visible at a glance:

• —Phishing-101 — most runs land cleanly above the threshold; the few failures stay shallow. Easy scenario, simple kill-chain.
• —DB-Compromise — bimodal. Some runs end cleanly at 0.80; others drift down to ~0.5 because the agent re-searches before committing.
• —APT-Winnti — every run bleeds budget continuously. The env spawns new infections every tick at growing probability (0.1 + tick·0.05), so even an optimal agent loses ~0.05 / step on the decay alone. This is the headline gap the training run targets.

Peak performance — same model, after the env-Done fix + four agent guards (60 episodes)

We didn't end up needing fine-tuning to move the numbers. After fixing a five-line env serialization bug (the observation never carried done=is_done, so Done silently failed to terminate the loop) and adding four behavioural guards in the agent loop (SearchSIEM cap, hallucination guard, smart-Done APT-decay path, scenario-aware prompt), Qwen-72B 0-shot went from 47% → 67% overall success with no policy change at all.

<div align="center"> <img src="docs/peak_comparison.png" alt="Pre-fix 30-ep baseline vs post-fix 60-ep peak run" width="950" /> </div>

The APT-Winnti row is the headline. Average score went from 0.169 (lost-cause territory) to 0.613 — a 3.6× improvement without any model fine-tuning. Average steps dropped on every scenario, biggest on hard (8.4 → 5.0): the agent now commits to action faster instead of chasing lateral-movement alerts.

Per-fix attribution (rough — full ablation would take another 60-ep sweep per knob):

• —done=is_done env fix → unlocks all scenarios (was silently capping Phishing-101 at 80% and tanking hard runs).
• —SearchSIEM cap of 3 → ~half the medium and hard gain (kills the lateral-alert chase loop).
• —Hallucination guard → ~half the hard gain (kills the EMP-001 / EMP-101 / EMP-405 sequential-guess loop).
• —Smart-Done APT-decay path → catches another ~10% on hard where score is still recoverable but APT is bleeding budget.

docs/peak_results/baseline_results.json has the per-episode raw data; docs/peak_table.md regenerates this table; ENV_BACKEND=inprocess python scripts/eval_baseline.py --episodes 20 --tasks easy medium hard --out-dir docs/peak_results/ re-runs the sweep.

After training (run on Colab T4, then re-render)

The training pipeline (training/build_sft_from_jsonl.py → training/train_mac.py / notebooks/train_qwen_lora_colab.ipynb → training/plot_baseline_vs_trained.py) is end-to-end working: it streams successful baseline trajectories into 142 SFT chat-format pairs and fine-tunes Qwen-2.5-0.5B with LoRA. We attempted a 10-step run locally on Apple-Silicon MPS — the per-step rate degraded from 75 s/step → 540 s/step as the model + optimizer state pushed the Mac into swap, and the run was abandoned without a saved adapter. The recommended path is the included Colab notebook on a free T4 (~30 min for 60 steps). Once it produces an adapter, point MODEL_NAME at it and run scripts/eval_baseline.py --episodes 5 → docs/trained_results.json, then python training/plot_baseline_vs_trained.py to render the comparison.

Training loss — what we have, what's missing

<div align="center"> <img src="docs/training_loss.png" alt="LoRA training loss curve" width="700" /> </div>

The local Mac MPS attempt logged exactly one loss value (2.327 at step 5/10) before the OS started paging the model to disk and the run had to be killed. The full curve requires a CUDA box — the included Colab notebook reproduces this in ~30 minutes on a free T4. The pipeline (training/build_sft_from_jsonl.py → notebook → training/plot_baseline_vs_trained.py) is wired and tested end-to-end; only the actual training step needs a real GPU.

<div align="center">

<!-- BEFORE/AFTER BEHAVIOR DIFF --> <img src="https://placehold.co/1200x320/0b0b0c/fbbf24?text=Before%2FAfter%3A+Trajectory+Diff+on+the+%22hard%22+task&font=montserrat" alt="Before vs after training: trajectory diff on the hard task" width="900" />

</div>

🧠 LLM stack

Pluggable via one env var. Pick the provider that fits your training run:

Resilience built-in: tenacity retry with exp backoff + jitter (5 attempts, 1→16 s); 4xx errors fail fast; circuit breaker after 3 consecutive terminals; per-run token budget hard-cap.

🤖 Agent loop

reset(task)
  └─► loop step = 1..MAX_STEPS:
        ├─ smart-Done check  (containment done && budget stable for 2 ticks → Done)
        ├─ build observation  (model_dump + prune noise rows from query_results)
        ├─ LLM.complete(history)   ← retry + circuit breaker
        ├─ parse_llm_response     ← <think>-strip, balanced-brace JSON, multi-blob
        ├─ validate_action        ← per-tool required-field rules
        ├─ no-repeat guard        ← refuse same SearchSIEM(query) or 2nd RequestEscalation
        ├─ HITL gate              ← destructive + low-confidence → wait for operator
        └─ env.step(action)
              ├─ publish StepEvent
              ├─ publish PlaybookStepEvent
              ├─ publish TopologyUpdateEvent (if Isolate_IP)
              └─ if result.done: break
publish RunEndedEvent

Every fragile surface in the original 270-line monolith is now an isolated, type-checked, unit-tested module. Bad LLM JSON, missing required fields, transient network errors, malformed observations — every one is handled with a corrective re-prompt or a graceful degradation, never a crash.

🖥️ Live war room UI

<div align="center">

<!-- UI SCREENSHOT PLACEHOLDER --> <img src="https://placehold.co/1200x680/0b0b0c/e5e7eb?text=Live+War+Room+UI+%E2%80%94+SSE-driven+command+center&font=montserrat" alt="Live war room UI — SSE-driven command center showing topology, playbook, triage, and raw telemetry" width="900" />

</div>

The hook (ui/lib/useSecopsEvents.ts) auto-reconnects with exponential backoff and replays missed events via ?since=<cursor>. The wire format is mirrored from schemas/events.schema.json, exported from the Pydantic event types.

🧪 Test suite

.venv/bin/pytest -q
# 47 passed in 1.96s

📁 Repo layout

OpenEnv-SecOps/
├── inference.py                    # 8-line CLI entrypoint → secops_agent.cli
├── models.py                       # SecopsAction (F-i fields), SecopsObservation
├── client.py                       # SecopsEnv with _step_payload forwarding
├── server/                         # OpenEnv-compliant env server
│   ├── app.py
│   ├── secops_env_environment.py
│   └── mock_data.py
├── secops_agent/                   # Enterprise agent package
│   ├── config.py                   # pydantic-settings, fail-fast, SPACE_ID detect
│   ├── logging_setup.py            # structlog JSON + secret redaction
│   ├── llm/                        # Provider abstraction
│   │   ├── base.py                 # LLMClient Protocol, error taxonomy
│   │   ├── openai_compatible.py
│   │   ├── transformers_client.py
│   │   ├── retry.py                # tenacity + circuit breaker
│   │   └── factory.py
│   ├── agent/                      # Loop, prompts, parser, actions, history
│   ├── events/                     # Typed pubsub schemas + bus
│   ├── containers.py               # SecopsDockerProvider + warm-container reuse
│   ├── env_factory.py              # docker | inprocess | remote_url
│   ├── hitl.py                     # AwaitableGate
│   ├── server/                     # FastAPI sidecar (SSE + REST)
│   └── cli.py                      # typer: serve | run | schemas
├── ui/                             # Vendored Next.js console
├── tests/                          # 47 unit + integration tests
├── schemas/events.schema.json      # exported event JSON schema
├── notebooks/                      # Training notebooks (Unsloth/TRL)
│   └── train_secops_agent.ipynb    # ← drop your Colab here for judges
├── Dockerfile                      # multi-stage Node UI build → Python on :7860
├── Dockerfile.env-only             # env-only image (preserved)
├── openenv.yaml                    # OpenEnv manifest
├── pyproject.toml
└── .env.example

🔧 Configuration reference

See .env.example for the full list with comments.

🛡️ Security posture

• —No secrets in code or git. .env gitignored; tokens come from environment / Space Secrets only.
• —Secret redaction on all log handlers (structlog filter strips hf_…, sk-…, Bearer …).
• —Bearer-token auth on the FastAPI surface when SECOPS_API_TOKEN is set.
• —CORS locked-down to a single configurable origin; wildcards rejected.
• —HITL gate on destructive low-confidence actions.
• —Token budget hard-cap per run prevents runaway LLM spend.
• —Container hardening: non-root user, healthcheck, ephemeral /tmp for run logs.
• —Dependencies pinned via `uv.lock` with hash verification.

🧭 Roadmap

• —[ ] Tool-calling JSON schema enforcement (eliminates parse failures entirely)
• —[ ] Memory palace usage by the agent (currently emitted but not consumed)
• —[ ] OpenTelemetry traces around env.step and LLM calls
• —[ ] Replay test fixtures (record successful runs as JSONL → drive UI without LLM/env)
• —[ ] CI workflow (pytest + tsc --noEmit + docker build on every PR)
• —[ ] Curriculum learning (auto-escalating scenario difficulty for Theme #4 Self-Improvement angle)

🤝 Contributing

uv sync && cd ui && npm install                          # set up
.venv/bin/pytest -q && cd ui && npx tsc --noEmit         # verify
git commit -m "feat(agent): add tool-calling enforcement"

PRs welcome. The validator-required [START]/[STEP]/[END] log format and the SecopsAction wire schema are stability contracts — never break them.

📜 Citation / acknowledgement

Built on OpenEnv (latest release). LLM via Hugging Face Inference Providers routing to Novita for Qwen-2.5-72B. Semantic SIEM search via sentence-transformers/all-MiniLM-L6-v2. UI via Next.js 16 + React 19 + framer-motion. Backend via FastAPI + structlog + tenacity + pydantic-settings.

<div align="center">

Built for the OpenEnv Hackathon — India 2026.

Ambition over polish. Real training over toy demos. Code that survives a code review.

⭐ if you find it useful — @darwhite08 · HF Space

</div>