Pandago/graphstrike-model-training
0
1# GraphStrike — Single Source of Truth2 3> Consolidates: `FINAL_SUMMARY.md`, `IMPLEMENTATION_COMPLETE.md`, `IMPLEMENTATION_STATUS.md`, `INFERENCE_UPDATE.md`, `PIPELINE.md`, `QUICKSTART.md`, `ROUND2_COMPLETE.md`, `ROUND2_STATUS.md`, `ROUND2_TRAINING_READY.md`, `server/ROUND2_FINAL_STATUS.md`, and the top-level `ROUND2_ARCHITECTURE.md` / `ROUND2_IMPLEMENTATION_PLAN.md` / `ROUND2_QUICK_REFERENCE.md` / `OpenEnv-Complete.md`.4>5> The HF-Space `README.md` is kept (it contains the YAML frontmatter Spaces needs). The per-directory `dashboard/README.md` describes only the local dashboard and stays with it.6 7---8 9## 1. What GraphStrike is10 11An OpenEnv-compatible RL environment. An LLM agent must identify the 10 members of a coordinated fake-account ring hidden inside a synthetic social network. Round 2 makes detection **platform-adaptive**:12 13- Each episode belongs to a platform (Instagram, Snapchat, X, LinkedIn, Reddit, … any name).14- A `PlatformPolicy` is **compiled from real transparency-report text** via a Bayesian threshold formula and cached per-platform.15- The high-signal account fields (`photo_reuse_score`, `bio_template_score`, `ip_cluster_id`) start hidden and are revealed only by explicit tool actions.16- Reward shape, FP penalty, grader score, and the moderation-decision package are all derived from the compiled policy rather than hardcoded.17 18A separate **shared evaluation runner** drives episodes deterministically and consults the LLM at exactly two decision points per suspicious account; six thin model shims plug in HF-router or Bedrock models against that runner.19 20---21 22## 2. Round 2 deltas (what changed vs Round 1)23 24| Area | Round 1 | Round 2 |25|---|---|---|26| Platform | — | `platform` field per episode; any name supported (env defaults to seed-parity Instagram/Snapchat) |27| Policy | hardcoded thresholds | `PlatformPolicy` compiled dynamically from transparency reports (Bayesian θ\*) with 30-day cache freshness and sanity checks |28| Signals | all visible at INSPECT | `photo_reuse_score`, `bio_template_score`, `ip_cluster_id` start at `0.0 / ""` and are revealed only by tool actions |29| Visible accounts | populated only on INSPECT | populated for every visible account from reset; tool reveals propagate immediately |30| Per-step reward | `null` for non-terminal steps | float delta of `self._score` returned every step |31| Actions | `inspect`, `investigate_network`, `flag`, `unflag`, `submit` | + `get_policy`, `reverse_image_search`, `analyze_bio`, `check_ip` |32| Reward shaping | terminal only | + `+0.20` first-action GET\_POLICY bonus, redundant-tool penalties, no-evidence flag deny |33| Submit response | `{observation, done, reward, message}` | + top-level `decision_package` and `grader_score` |34| Eval | one monolithic `qwen_test_judge_eval.py` per model | shared `_round2_runner.py` + 6 thin shims, two LLM decision points per account |35 36Platform assignment is deterministic in the env: `seed % 2 == 0 → Instagram`, else Snapchat. The eval runner remaps seeds so any requested platform actually fires (`--platform Instagram` forces even seeds, `--platform Snapchat` forces odd).37 38---39 40## 3. End-to-end policy flow (from transparency report to gradient signal)41 42This is the spine of Round 2. Every other component reads from this pipeline.43 44```45 (ONE-TIME / OFFLINE)46 transparency-report URLs policy_cache/47 ──────────────────────── ─────────────48 │ │49 ▼ ▲50 Tavily search │51 query: "{platform} fake account content │52 policy enforcement 2024 2025" │53 │ │54 ▼ │55 Groq Llama-3.1-8B extraction │56 → {base_rate π, fn_cost_signal, fp_cost_signal, │57 harm_weight, primary_signal, confidence} │58 │ │59 ▼ │60 sanitize_pi() — clamp [0.0005, 0.05] │61 (>0.05 ⇒ "enforcement rate misread", clamp + warn) │62 │ │63 ▼ │64 compute_threshold(π, fn_signal, fp_signal, hw) │65 ──────────────────────────────────────────────── │66 C_fn = FN_COST_MAP[fn_signal] │67 C_fp = FP_COST_MAP[fp_signal] │68 θ_raw = C_fn·π / [C_fn·π + C_fp·(1−π)] │69 θ* = clamp(θ_raw / harm_weight, 0.01, 0.95) │70 fp_penalty_weight = C_fp │71 │ │72 ▼ │73 PlatformPolicy(threshold=θ*, base_rate=π, │74 fn/fp_cost_signal, harm_weight, │75 primary_enforcement_signal, │76 fp_penalty_weight=C_fp, │77 confidence, sources, used_fallback) ────┘78 │79 ▼ sanity_check_policy() — surfaces warnings80 ▼ (high θ*, suspicious π, low confidence, bad signal name)81 ▼82 cached to policy_cache/{platform}.json83 │84=====================│=====================85 │ (PER EPISODE — RUNTIME)86 ▼87 client.reset(task, seed)88 env.platform = "Instagram"89 env._policy = get_policy("Instagram") ◄── reads cached JSON90 │ (recompiles if >30 days old)91 ▼92 deterministic step 0: GET_POLICY (free, +0.20 first-action bonus)93 message: "Policy compiled: Platform: Instagram |94 Threshold: 0.369 | Primary Signal: photo_reuse | FP Penalty: 0.1x | …"95 │96 ▼97 runner._policy_from_message() → policy dict {threshold, primary_signal, fp_weight}98 │99 ▼100 per suspicious account, sorted by risk_score desc:101 INSPECT (deterministic)102 INVESTIGATE_NETWORK if risk ≥ 0.80 (deterministic, once)103 104 ┌─ DP1 (LLM) ─────────────────────────────┐105 │ prompt includes platform, primary_signal,│106 │ θ*, revealed-vs-None signals, budget │107 │ → "reverse_image_search" / "analyze_bio" │108 │ / "check_ip" / "done" │109 └──────────────────────────────────────────┘110 ↓ (loop until "done" or signals sufficient)111 112 ┌─ DP2 (LLM) ─────────────────────────────┐113 │ prompt includes revealed signals, │114 │ θ*, fp_penalty=C_fp, running tp/fp count │115 │ → "flag" / "skip" │116 └──────────────────────────────────────────┘117 │118 ▼119 SUBMIT (deterministic)120 reward = tp·1.0 − fp·C_fp − fn·0.3 + bonuses − penalties121 ▲122 └── platform-specific via fp_penalty_weight123 grader_score and decision_package surfaced at top level of /step response.124```125 126**Two views of the same policy:**127- `θ*` is in the **prompt** at DP1/DP2 → the LLM conditions on it.128- `C_fp` (= `fp_penalty_weight`) is in the **terminal reward** → the LLM is graded against it.129 130Both come from the same compile-time computation; they cannot drift apart.131 132---133 134## 4. Policy Compiler (`server/policy_compiler.py`)135 136### 4.1 Formula137 138```139θ_raw = C_fn · π / [C_fn · π + C_fp · (1 − π)]140θ* = clamp(θ_raw / harm_weight, 0.01, 0.95)141fp_penalty_weight = C_fp142```143 144Action rule the threshold serves: **FLAG if `risk_score ≥ θ*`**.145 146`θ_raw` is the share of expected cost coming from missed fakes. Higher `C_fn` or higher base rate → higher `θ_raw` → lower threshold (the agent should flag more aggressively when misses are expensive).147 148`harm_weight > 1` strict (lowers θ\*); `harm_weight < 1` lenient (raises θ\*).149 150> **History note.** The original spec used `θ_raw = C_fp(1−π) / [C_fp(1−π) + C_fn·π]` — the *complementary* probability. With small π that formula collapses to `≈ 1` for every platform (π is the bottleneck, not the costs). Audit on 2026-04-25 confirmed this was a formula-direction error; the orientation above is correct for our action rule.151 152### 4.2 Cost maps153 154```python155FN_COST_MAP = {"low": 0.5, "medium": 1.0, "high": 2.0, "critical": 4.0}156FP_COST_MAP = {"low": 0.1, "medium": 0.5, "high": 1.5}157```158 159Signals are extracted from policy text by an LLM and constrained to these keys (defaults `high` / `medium` if absent or invalid).160 161### 4.3 Extraction inputs162 163| Field | Source | Sanitization |164|---|---|---|165| `base_rate` (π) | LLM extraction from transparency report | `sanitize_pi`: clamp to `[0.0005, 0.05]`; >0.05 logs *"likely enforcement rate misread, clamped"*. The prompt also instructs the LLM to return `0.005` if it sees an enforcement rate or no prevalence figure. |166| `fn_cost_signal` | LLM extraction | invalid → `high` |167| `fp_cost_signal` | LLM extraction | invalid → `medium` |168| `harm_weight` | LLM extraction | non-numeric → `1.0` |169| `primary_enforcement_signal` | LLM extraction | None / blank / non-string → `photo_reuse` |170| `confidence` | LLM extraction | non-numeric → `0.0` |171 172### 4.4 Tavily query (generic, platform-agnostic)173 174```python175query = f"{platform} fake account content policy enforcement 2024 2025"176```177 178The previous query was Meta/Instagram-specific; the generic form works for **any** platform name. Domain filtering (`is_high_signal_source`) was removed for the same reason — it gated to meta.com/snap.com domains.179 180### 4.5 Caching & freshness181 182- Cached at `policy_cache/{platform_lowercase}.json`.183- Entries older than `CACHE_TTL_DAYS = 30` are treated as stale and recompiled.184- `compile_policy(platform, use_cache=True)` is the runtime entry; `--use-cache` flag controls CLI behavior (default re-compile when invoked from CLI).185 186### 4.6 Fallbacks187 188- `FALLBACK_POLICIES` provides hardcoded params for Instagram / Snapchat. Any other platform falls back to `GENERIC_FALLBACK` (π=0.005, fn=high, fp=medium, hw=1.0).189- Fallback policies set `used_fallback=True` (a new field on `PlatformPolicy`).190- The **threshold value** in fallbacks is computed via the same formula — there is no hardcoded threshold in the policy compiler anymore.191 192### 4.7 Sanity check (`sanity_check_policy`)193 194After every compile, the compiler prints warnings for any of:195 196| Trigger | Meaning |197|---|---|198| `θ* > 0.90` | agent will almost never flag — check fn_cost extraction |199| `θ* < 0.005` | agent will flag nearly everything — check fp_cost extraction |200| `base_rate > 0.05` | likely enforcement-rate misread |201| `confidence < 0.60` | low extraction quality; consider falling back |202| `primary_signal ∉ {photo_reuse, bio_template, ip_cluster, behavior}` | not a known tool action |203 204Sanity check **does not block compilation**; it surfaces issues so an operator can review before running eval.205 206### 4.8 CLI207 208```bash209python -m server.policy_compiler --platform <Name> # always recompile210python -m server.policy_compiler --platform <Name> --use-cache211```212 213### 4.9 Currently compiled policies214 215| Platform | π | fn_signal | fp_signal | hw | θ\* | C_fp | confidence | used_fallback |216|------------|------:|-----------|-----------|----:|------:|-----:|-----------:|--------------:|217| X | 0.005 | high | low | 1.0 | 0.091 | 0.10 | 0.80 | False |218| Instagram | 0.030 | critical | low | 1.5 | 0.369 | 0.10 | 0.80 | False |219| Snapchat | 0.005 | low | low | 1.0 | 0.025 | 0.10 | 0.50 ⚠ | False |220| LinkedIn | 0.005 | critical | low | 1.0 | 0.167 | 0.10 | 0.80 | False |221| Reddit | 0.005 | low | low | 1.0 | 0.025 | 0.10 | 0.50 ⚠ | False |222 223Snapchat and Reddit currently raise the *low confidence* sanity warning — extraction is noisy on those transparency reports. Consider forcing the fallback path before training on them.224 225---226 227## 5. Hidden-signal architecture228 229Episode JSON stores hidden signals at episode level, not per account:230 231```json232{233 "episode_id": "easy_042_Instagram",234 "platform": "Instagram",235 "hidden_signals": {236 "photo_reuse": {"acc_0001": 0.87, ...},237 "bio_template": {"acc_0001": 0.72, ...},238 "ip_cluster": {"acc_0001": "ip_gang_42", ...}239 }240}241```242 243`account.features` start with `photo_reuse_score = 0.0`, `bio_template_score = 0.0`, `ip_cluster_id = ""`. Tool handlers copy from `ep["hidden_signals"]` into `account.features` and refresh the cached profile so subsequent observations carry the revealed value.244 245> **Known limitation.** `generator.py` accepts a `platform` arg but currently produces identical hidden-signal distributions for every platform. Platform conditioning is therefore purely *prompt-side* — the LLM learns to read θ\* and C_fp from the prompt and reward, not to recognize platform-specific data shape. Parametrizing the generator by platform is a separate follow-up.246 247---248 249## 6. Scoring (`server/scoring.py`)250 251Stateless risk functions (kept from Round 1): `compute_node_risk`, `compute_behavior_risk`, `compute_graph_risk`, `compute_hub_legitimacy`, `compute_fake_risk`.252 253Round 2 additions:254- `compute_weighted_fake_risk(..., primary_signal)` boosts the platform's primary signal (node risk +0.15 for content signals; behavior risk +0.15 for `ip_cluster`).255- `classify_risk(fake_risk, threshold)` accepts platform threshold.256- `grader_score(tp, fp, fn, steps, max_steps, threshold, fp_penalty_weight)` adds `0.05 × (1 − threshold)` to reward stricter platforms.257 258Win conditions (unchanged from Round 1): easy/medium `recall ≥ 0.8, precision ≥ 0.7`; hard `recall ≥ 0.9, precision ≥ 0.8`.259 260---261 262## 7. Tool-action contracts263 264| Action | Step cost | Score delta | Reveals | Notes |265|---|---|---|---|---|266| `GET_POLICY` | 0 | `+0.20` once (first action) | — (returns `PlatformPolicy` summary in `message`) | Free; bonus only fires on `_action_count == 1` |267| `INSPECT` | 1 | `−0.01` | full profile, edges | needed before any DP1/DP2 logic |268| `REVERSE_IMAGE_SEARCH` | 1 | `−0.01` (`−0.05` if redundant) | `photo_reuse_score` | sets `account.features.photo_reuse_score` |269| `ANALYZE_BIO` | 1 | `−0.01` (`−0.05` if redundant) | `bio_template_score` | sets `account.features.bio_template_score` |270| `CHECK_IP` | 2 | `−0.02` (`−0.10` if redundant) | `ip_cluster_id` + cluster-size message | heaviest tool; only worth it for shared_ip ≥ 5 |271| `INVESTIGATE_NETWORK` | 2 | `−0.02` | 2-hop expansion + SUSPECT cascade | unchanged from Round 1 |272| `FLAG` | 0 | `−0.15` if no evidence (deny) | dual SUSPECT cascade (follow-graph + IP) | "no evidence" = not inspected AND no tool used on the account |273| `UNFLAG` | 0 | 0 | — | unchanged |274| `SUBMIT` | 0 | terminal-reward formula (§ 8) | end episode | also surfaces `decision_package` and `grader_score` at top level |275 276All tool handlers validate `acc_id in self._accounts`, refresh the cached profile, and force `_do_submit(forced=True)` if max steps were consumed.277 278---279 280## 8. Reward shape (per-step deltas + terminal)281 282Per-step delta is now visible on every `/step` response: it is `round(self._score - self._last_score, 4)`. `terminal_reward` overrides the delta on the SUBMIT step so the caller sees the full episode reward there.283 284### 8.1 Per-step shaping (visible immediately)285 286```287+0.20 GET_POLICY as first action (once per episode)288-0.01 per inspect / reverse_image_search / analyze_bio (time cost)289-0.02 per check_ip / investigate_network (time cost)290-0.05 per redundant reverse_image_search / analyze_bio291-0.10 per redundant check_ip292-0.15 blind FLAG (no inspect, no tool used on account) ← deny + penalty293```294 295### 8.2 Terminal reward at SUBMIT296 297```298reward = tp · 1.0299 − fp · self._policy.fp_penalty_weight (= C_fp; varies per platform)300 − fn · 0.3301 + 5.0 if recall ≥ win_recall AND precision ≥ win_precision302 + 3.0 if tp == 10 (perfect recall)303 + 2.0 if partial win (recall met, precision missed)304 + 1.0 if SUBMIT with ≥ 50% steps remaining305 + 2.0 if Instagram and precision ≥ 0.95306 + 2.0 if Snapchat and recall ≥ 0.95307 − 1.0 × evasion_count (hard task only)308 − 2.0 if forced SUBMIT (ran out of steps)309 − 0.15 × |unsupported_flags| (flags with no revealed signals at submit time)310```311 312Note: `fp_penalty_weight` is platform-specific and is the principal lever the policy compiler pulls. Same FP behavior costs more on X (1.5) than on Instagram/Snapchat (0.1).313 314---315 316## 9. Schemas (OpenEnv-compliant)317 318### 9.1 Models319 320- `FakeGangAction`: `action_type: ActionType`, `account_id: Optional[str]`321- `FakeGangObservation`: `done`, `reward` (per-step delta or terminal), `visible_accounts[AccountProfile]` (now populated for every visible id), `visible_account_ids`, `flagged_ids`, `inspected_ids`, `graph_edges`, `steps_remaining`, `evasion_triggered`, `evasion_count`, `task`, `message`, `suspect_ids`, **`platform`**322- `FakeGangState`: `episode_id`, `step_count`, `task`, `score_so_far`, `evasion_count`, `network_size`, `gang_size`, `episode_seed`, **`platform`**323- `PlatformPolicy`: `platform`, `threshold`, `base_rate`, `fn_cost_signal`, `fp_cost_signal`, `harm_weight`, `primary_enforcement_signal`, `fp_penalty_weight`, `sources`, `confidence`, `compiled_at`, **`used_fallback`**324 325### 9.2 `StepResponse` (HTTP)326 327```json328{329 "observation": { ... },330 "done": <bool>,331 "reward": <float | null>,332 "message": "...",333 "decision_package": { ... } | null, // populated after SUBMIT334 "grader_score": <float> | null // populated after SUBMIT, sourced from decision_package335}336```337 338`decision_package` (after SUBMIT) carries:339- `platform`, `flagged_accounts[]`, `recommended_action ∈ {queue_for_review, temporary_hold, scheduled_ban, batch_takedown}`340- `evidence_summary`: `flagged`, `revealed_photo_reuse`, `revealed_bio_template`, `revealed_ip_cluster`, `unsupported_flags[]`341- `policy_rationale`: textual explanation including θ\*, primary signal, FP penalty, observed precision/recall342- `tp`, `fp`, `fn`, `precision`, `recall`, `reward`, `grader_score`343 344The terminal `message` also embeds the keywords `flagged_accounts`, `evidence_summary`, `policy_rationale`, `grader_score` for callers that grep the message string.345 346---347 348## 10. HTTP API (`server/app.py`)349 350| Endpoint | Method | Notes |351|---|---|---|352| `/health` | GET | `{"status":"healthy"}` |353| `/reset` | POST | `{task, seed, episode_id}` → `StepResponse` |354| `/step` | POST | `FakeGangAction` body → `StepResponse` (per-step reward delta + decision_package + grader_score on SUBMIT) |355| `/state` | GET | Current `FakeGangState` |356| `/tasks` | GET | Task list + Round 2 action_schema (9 actions) |357| `/grader` | GET | Normalized [0,1] score; requires SUBMIT first |358| `/metadata` | GET | HF Spaces metadata |359| `/schema` | GET | Pydantic JSON schemas |360| `/mcp` | POST | MCP JSON-RPC for tools/list |361| `/baseline` | POST | Runs rule-based baseline on all 3 tasks |362| `/` | GET | Gradio playground |363 364`openenv.yaml` action schema mirrors all nine action types (Round 1 five plus the four Round 2 tools).365 366---367 368## 11. Evaluation runner (`eval-models/_round2_runner.py`)369 370### 11.1 Outer loop (deterministic)371 372```373reset(task, seed)374 ↓375GET_POLICY (step 0) ← always; bonus +0.20376 ↓377loop over visible accounts sorted by (suspect_flag, risk_score) desc:378 INSPECT if not yet inspected379 INVESTIGATE_NETWORK if risk_score ≥ 0.80 (once per account, ≥5 steps left)380 381 DP1 loop (LLM) — pick a tool or "done"382 reverse_image_search | analyze_bio | check_ip | done383 stops on "done", missing budget, or photo + bio both revealed384 385 DP2 (LLM) — flag-or-skip386 flag → env.step(FLAG)387 skip → leave alone, move to next account388 ↓389SUBMIT390```391 392Stops early when `done` is signaled, `steps_remaining ≤ 1`, or `max_accounts_per_episode = 15` accounts have been processed.393 394### 11.2 The two LLM decision points395 396**DP1 — tool selection** prompt includes:397- `platform`, `primary_signal`, `θ*`398- `account_id`, `risk_score`, `hub_legitimacy`399- Each revealed signal value or `None`400- `steps_remaining`, tool costs401 402**DP2 — flag decision** prompt includes:403- All revealed signals for the account404- `θ*`, `fp_penalty = C_fp`405- Running `flagged / 10`, `steps_remaining`406 407Each prompt asks for **exactly one token** so parsing is robust. Invalid completions are counted in `dp1_invalid` / `dp2_invalid` for QA.408 409### 11.3 Per-episode JSONL log410 411`eval-models/results/{model}_{platform}_results.jsonl` — one line per episode:412 413```json414{415 "model": "Bedrock/qwen.qwen3-next-80b-a3b",416 "platform": "Instagram",417 "task": "easy", "seed": 0,418 "episode_id": "easy_000_Instagram",419 "threshold": 0.369, "primary_signal": "photo_reuse",420 "steps_taken": 14, "inspected": 5,421 "tool_calls": {"reverse_image_search": 5, "analyze_bio": 4, "check_ip": 1,422 "get_policy": 1, "investigate_network": 1},423 "flagged": 7,424 "dp1_calls": 12, "dp2_calls": 5, "dp1_invalid": 0, "dp2_invalid": 0,425 "reward": 4.32, "grader_score": 0.71,426 "final_message": "...", "wall_seconds": 23.4427}428```429 430### 11.4 Public entry point431 432```python433from _round2_runner import run_evaluation434run_evaluation(435 model_name="qwen-72b",436 call_llm=lambda prompt: ..., # injectable adapter437 platform="Instagram",438 base_url="http://localhost:7860",439 tasks=["easy", "medium", "hard"],440 seeds=[0, 1, 2],441)442```443 444The runner remaps requested seeds to the env's parity rule so `--platform Instagram` actually runs Instagram episodes (`even`) and `--platform Snapchat` runs Snapchat (`odd`). Other platform names pass seeds through unmodified (env then falls back to its parity default for that seed).445 446### 11.5 Import-order safety447 448The runner unconditionally inserts the project root at `sys.path[0]` and evicts any cached `models` / `client` modules so a stale copy in `~/.local/lib/python3.12/site-packages` cannot win. If your shim raises `ActionType has no attribute GET_POLICY`, that means the safety insert was skipped — verify you are running today's runner.449 450---451 452## 12. Model shims (`eval-models/{qwen,gemma,deepseek,llama,mistral,nvidia}_test_judge_eval.py`)453 454Each shim is ~30 lines. It declares the model identifiers and delegates to the runner via `_llm_adapters.make_caller`:455 456| Shim | HF model | Bedrock model |457|---|---|---|458| qwen | `Qwen/Qwen2.5-72B-Instruct` | `qwen.qwen3-next-80b-a3b` |459| gemma | `google.gemma-3-12b-it` | same |460| deepseek | `deepseek.v3.2` | same |461| llama | `meta.llama4-scout-17b-instruct-v1:0` | same |462| mistral | `mistral.ministral-3-8b-instruct` | same |463| nvidia | `nvidia.nemotron-super-3-120b` | same |464 465`_llm_adapters.py` exposes `make_hf_caller(model)`, `make_bedrock_caller(model_id)`, and a unified `make_caller(backend, hf_model, bedrock_model)`. Both backends strip `<think>...</think>` reasoning blocks and retry up to 3× with exponential backoff.466 467### Usage468 469```bash470# HF router (needs HF_TOKEN)471python eval-models/qwen_test_judge_eval.py --url http://localhost:7860 --platform Instagram472 473# AWS Bedrock (needs AWS_* env vars)474python eval-models/qwen_test_judge_eval.py --bedrock --url http://localhost:7860 --platform Snapchat \475 --tasks easy medium --seeds 0 1 2476```477 478---479 480## 13. Files that matter481 482**Source of truth (read first):**483- `reference.md` — this file484- `models.py` — data schemas (`PlatformPolicy.used_fallback` is new)485- `server/policy_compiler.py` — Bayesian θ\*, sanity check, generic Tavily, 30-day cache486- `server/environment.py` — reset/step/state, tool handlers, per-step reward delta, no-evidence flag deny, decision package487- `server/app.py` — `StepResponse` with top-level `decision_package` and `grader_score`488- `server/scoring.py` — risk/grader math489- `server/generator.py` — episode generation, `hidden_signals`490- `eval-models/_round2_runner.py` — deterministic loop + DP1/DP2491- `eval-models/_llm_adapters.py` — HF + Bedrock callers492- `eval-models/{model}_test_judge_eval.py` — six thin shims493- `openenv.yaml` — action schema mirrors all 9 actions494- `check.sh` — 12-step Round 2 system check (server side)495 496**Operational:**497- `policy_cache/{platform}.json` — compiled policies (delete to force recompile)498- `episodes/{task}_{seed}.json` — generated episodes (regenerate with `python -m server.generator`)499- `eval-models/results/{model}_{platform}_results.jsonl` — per-episode eval logs500 501**Round 1 still functional:**502- `agent/train.py`, `agent/policy.py`, `agent/memory.py`, `agent/reflection.py`, `agent/hybrid_policy.py`503- `inference.py`, `bedrock_model.py`, `client.py`504- `validate.py`, `test_round2.py`505 506---507 508## 14. Quickstart509 510```bash511# 1. Install512cd fake_gang_env513uv sync # or: pip install -r requirements.txt514 515# 2. Compile / refresh platform policies (one-time, then per ≥30 days)516python -m server.policy_compiler --platform Instagram517python -m server.policy_compiler --platform Snapchat518python -m server.policy_compiler --platform X519python -m server.policy_compiler --platform LinkedIn520 521# 3. (Re)generate episodes522python -m server.generator523 524# 4. Start the env server525python -m uvicorn server.app:app --port 7860526 527# 5. End-to-end system check (12 verifications)528bash check.sh529 530# 6. Run a model shim against the live server531export HF_TOKEN=... # or AWS_*532python eval-models/qwen_test_judge_eval.py \533 --url http://localhost:7860 \534 --platform Instagram \535 --tasks easy medium hard \536 --seeds 0 1 2537# Logs: eval-models/results/Qwen_Qwen2.5-72B-Instruct_instagram_results.jsonl538```539 540Docker:541```bash542docker build -f server/Dockerfile -t graphstrike .543docker run -p 7860:7860 -v $(pwd)/memory:/app/memory -v $(pwd)/runs:/app/runs graphstrike544```545 546---547 548## 15. System check (`check.sh`)549 550Twelve numbered checks against a running server at `http://localhost:7860`:551 552| # | Check | Pass criterion |553|---|---|---|554| 1–4 | health, /tasks, /reset, /step GET_POLICY | endpoints respond; action schema lists 9 types; threshold appears in message |555| 5 | INSPECT first visible account | profile returned; account_id is real (extracted from `visible_account_ids` *before* inspect) |556| 6 | REVERSE_IMAGE_SEARCH | `photo_reuse_score > 0` for that account in `observation.visible_accounts[*]` |557| 7 | ANALYZE_BIO | `bio_template_score > 0` |558| 8 | CHECK_IP | message reports cluster, `shared_ip_count` populated |559| 9 | GET_POLICY first-action bonus | per-step `reward ≥ 0.15` |560| 10 | redundant tool penalty | second `reverse_image_search` reward < first |561| 11 | blind FLAG penalty | flag without prior inspect/tool → reward `≤ −0.10` |562| 12 | full episode | submit response carries the four decision-package keywords + non-null `grader_score` |563 564CHECK 5–8 read from `observation.visible_accounts[*]` rather than a non-existent top-level `profile` field — the prior version of `check.sh` had that bug.565 566---567 568## 16. Bug fixes shipped 2026-04-25569 570| # | File | Symptom | Root cause | Fix |571|---|---|---|---|---|572| 1 | `server/environment.py` | `reward: null` on every non-terminal step | `_make_observation` only set `terminal_reward` | Track `_last_score`; return `score - _last_score` as per-step delta |573| 2 | `server/environment.py` | `visible_accounts: []` until INSPECT | observation included only `_profiled` | Build a profile for every `_visible_id` (cached for inspected, fresh otherwise). Tool reveals propagate because `_build_profile` reads from `account.features` which the tool handlers update. |574| 3 | `server/environment.py` | Tool reveals invisible to caller | covered by Bug 2 | — |575| 4 | `server/environment.py` | GET_POLICY +0.20 not visible | accumulated into `_score` but `_make_observation` never returned it | covered by Bug 1 |576| 5 | `server/environment.py`, `server/app.py` | submit response missing decision-package keywords + `grader_score` | message lacked the literal keywords; StepResponse only had four fields | enrich submit message; add `decision_package` and `grader_score` as top-level fields on StepResponse |577| 6 | `server/environment.py` | blind FLAG (no inspect, no tool) returned 0 reward | submit-time `unsupported_flags` only fires at SUBMIT | `_do_flag` now denies blind flags immediately with `−0.15` |578| 7 | `eval-models/_round2_runner.py` | `ActionType has no attribute GET_POLICY` when running shims | `if _PARENT not in sys.path` guard skipped the insert because path was already present at lower priority; site-packages `models.py` won | Insert `_PARENT` at index 0 unconditionally; evict cached `models`/`client` from `sys.modules` |579| 8 | `check.sh` | acc\_000 hardcoded; profile field read from wrong path | script bugs | extract real `account_id` from `observation.visible_account_ids` *before* CHECK 5; read profiles from `observation.visible_accounts[*]` |580| — | `server/policy_compiler.py` | θ\* always ≈ 0.95 | formula direction inverted (computed FP-cost share, not FN-cost share) | `θ_raw = C_fn·π / [C_fn·π + C_fp·(1−π)]` |581| — | `server/policy_compiler.py` | enforcement-rate misreads (e.g. Snap π=0.262) | LLM confusion between "% removed" and "% prevalence" | `sanitize_pi` clamp `[0.0005, 0.05]` + warning; extraction prompt explicitly disambiguates |582| — | `server/policy_compiler.py` | crash on Pydantic validation when LLM returned `None` for `primary_enforcement_signal` | strict typing | coerce None / blank to `photo_reuse`; same for `confidence` |583 584---585 586## 17. Sanity rules for adding a new platform587 588After running `python -m server.policy_compiler --platform <Name>`:589 590| Property | Acceptable range | Action if outside |591|---|---|---|592| `threshold` | `[0.005, 0.90]` | review — likely cost-signal extraction issue |593| `base_rate` | `[0.0005, 0.05]` | review — likely enforcement-rate misread |594| `confidence` | `≥ 0.60` | force fallback or improve sources |595| `primary_signal` | one of `{photo_reuse, bio_template, ip_cluster, behavior}` | coerced to `photo_reuse` |596| `used_fallback` | match expectation | ensure Tavily/Groq keys are set if False expected |597 598Cross-platform ordering is **not** an invariant. Any platform may land anywhere on the [0.01, 0.95] θ\* scale depending on its actual policy.599 600---601 602## 18. Outstanding (optional) work603 6041. **Platform-specific episode generation** — `generate_episode` accepts a `platform` arg but produces identical hidden-signal distributions. Parametrize π, signal strengths, and evasion behavior per platform for richer training data.6052. **TRL/GRPO trainer wrapper** — runner produces `(prompt, completion)` pairs at DP1/DP2 and per-step rewards. Threading these into a TRL `DataCollator` is the next step (training-side scope, not part of this readiness pass).6063. **Force-fallback flag on the CLI** — convenient way to ignore Tavily and use hardcoded params when sanity check raises low-confidence warnings.6074. **`hybrid_policy.py` platform-aware upgrade** — Round-1 rule engine still uses fixed `_THRESHOLDS`; could read `env._policy.threshold`. Low priority since `agent/train.py` and the eval runner are independent.6085. **Dashboard** — `dashboard/DASHBOARD_SPEC.md` describes a React + D3 demo; not required.609 610---611 612## 19. Design decisions (kept from earlier docs, condensed)613 614- **Hidden signals at episode level, not account level** — easier to track revelation, cleaner rollback between episodes.615- **Platform assignment by seed parity (env)** — reproducible without extra RNG state; eval runner remaps seeds when `--platform` is requested.616- **Bayesian θ\*** — principled, explainable, varies sensibly when policy text changes. Action rule is `FLAG if risk ≥ θ*`.617- **Asymmetric tool costs** — CHECK_IP is 2× to force the agent to use cheap signals first.618- **Cached policies + 30-day TTL** — hackathon-demo viable without network; live recompile on staleness.619- **Two LLM decision points** — keeps the LLM's job focused (tool-pick + flag/skip) and makes (prompt, completion, reward) tuples cleanly attributable for future RL training.620- **Top-level `decision_package` + `grader_score`** — callers shouldn't have to grep the message string for the four submission fields.621 622---623 624## 20. Known tests / validation625 626- `bash check.sh` — 12-step end-to-end against a running server (Round 2 system check).627- `test_round2.py` — 9-stage Python test against `server/environment.py`.628- `validate.py` — 24 HTTP validator checks against a running server.629- `eval-models/{model}_test_judge_eval.py` — judge model vs. environment scoring with two-decision-point loop.630 631All four were verified against the current tree on 2026-04-25.632