AxeH666/meta_RL_mod

1# ModGuard-RL — Cursor Rules2# Spec v1.5.0 is LOCKED. These rules are absolute.3 4## IDENTITY5You are implementing a real-world RL environment for a competitive hackathon.6The spec is mathematically verified and frozen. Your job is precise implementation,7not design. Never suggest changing spec decisions. If something seems wrong, say so8and wait for confirmation before touching anything.9 10## CRITICAL — NEVER DO THESE11- Never use global random, random.random(), random.choice(), or any random call12  outside of self.rng. ALL randomness uses self.rng = random.Random(seed) exclusively.13- Never add fields to ModGuardObservation, ModGuardAction, or ModGuardState that14  are not in the spec.15- Never change enum values in ContentCategory, RiskLevel, PlatformContext,16  ActionType, GTLabel, or Stage.17- Never modify the grade formula weights: correctness×0.45, process×0.25,18  hint×0.20, speed×0.10. These are locked.19- Never modify the correctness table values.20- Never let path_penalty_incurred reset to False once it is True within an episode.21- Never place the Dockerfile inside server/. It belongs at project root only.22- Never import openai or use the OpenAI API key. Only HF token + HF router.23- Never add an LLM call inside environment.py. The environment is fully deterministic.24- Never use wildcard imports (from x import *) in any file.25 26## SPEC — LOCKED VALUES (do not compute, do not infer, copy exactly)27 28### Grade formula29grade = correctness×0.45 + process×0.25 + hint×0.20 + speed×0.1030 31### Correctness table [GT][terminal_action]32             approve  remove  escalate  legal_hold33approve       1.0     0.5     0.1       0.034remove        0.5     1.0     0.3       0.135legal_hold    0.0     0.1     0.5       1.036 37### Process score38budget_violated=True → 0.0 (return immediately)39diff=0 → 1.0 | diff=1 → 0.6 | diff=2 → 0.240path_penalty_incurred=True → min(score, 0.4)41 42### Optimal steps43GT=approve      any risk     → 144GT=remove       any risk     → 145GT=legal_hold   non-critical → 146GT=legal_hold   critical     → 347 48### Speed score49steps=1 → 1.0 | steps=2 → 0.6 | steps=3 → 0.250 51### Hint score52hint=None                                    → 0.553hint==terminal_action AND hint==GT           → 0.854hint==terminal_action AND hint!=GT           → 0.055hint!=terminal_action AND hint==GT           → 0.356hint!=terminal_action AND hint!=GT           → 1.057 58### ai_confidence_score59base = rng.uniform(0.7, 1.0)60easy:   score = base61medium: score = clamp(base + rng.uniform(-0.2, 0.2), 0.0, 1.0)62hard:   score = 1.0 - base  ← always, no probability split63 64### human_reviewer_hint65hard:   always None66easy:   always ActionType(ground_truth.value)67medium: correct with prob 0.6, else random wrong ActionType68 69## PATH PENALTY — 5 TRIGGERS (sticky, never resets)70TRIGGER 1: stage=escalation_review, action=escalate → set True, TERMINAL71TRIGGER 2: stage=escalation_review, action=legal_hold, risk!=critical, GT!=legal_hold → set True, TERMINAL72TRIGGER 3: stage=legal_review, action=escalate → set True, TERMINAL73TRIGGER 4: stage=legal_review, action=legal_hold, GT!=legal_hold → set True, TERMINAL74TRIGGER 5: entering step 3, action_history[0]==legal_hold → set True immediately on entry75 76## BUDGET VIOLATION GUARD77Check before any decrement:78  if action == escalate AND escalation_budget == 0:79      budget_violated = True80      # do NOT decrement further — budget stays at 081 82## OBSERVATION UPDATE BETWEEN STEPS83Only these three fields change between steps:84  - step_number (increment)85  - stage (per state machine)86  - reviewer_overturn_rate (resample: clamp(rng.uniform(0,1) + rng.uniform(-0.15,0.15), 0.0, 1.0))87All other fields are identical to the reset observation. Never resample them.88reviewer_overturn_rate is always None at step 1. Always non-None at step 2+.89 90## REWARD POLICY91non-terminal: reward=0.0, done=False92terminal:     reward=grade(), done=True93Step penalties are NOT subtracted from reward. They affect process_score only.94 95## FILE STRUCTURE — EXACT, DO NOT CHANGE96modguard_rl/97├── Dockerfile          ← ROOT level, never inside server/98├── openenv.yaml99├── pyproject.toml100├── inference.py101├── README.md102├── server/103│   ├── app.py104│   ├── environment.py105│   └── models.py106└── client/107    └── client.py108 109## DOCKERFILE REQUIREMENTS110ENV ENABLE_WEB_INTERFACE=true must be present.111Dockerfile must be at project root.112 113## INFERENCE SCRIPT REQUIREMENTS114Must use HF router base URL, not api.openai.com.115API key loaded from HF_TOKEN environment variable only.116Model: Qwen/Qwen2.5-72B-Instruct (or similar via HF router).117Must run end-to-end with: uv run inference.py118Reward values logged per episode — must show diversity across runs.119 120## CODE QUALITY RULES121- Type hints on every function signature.122- No bare except clauses.123- No print debugging left in final files.124- Every method that touches state must be called only after reset().125- _grade() and _process_score() are private methods, never called externally.126- get_state() returns a copy or the live ModGuardState — never None.127 128## WHEN GENERATING CODE1291. Read the relevant existing file first before writing or editing.1302. Write complete files — no "rest remains the same" placeholders.1313. After writing, state explicitly which spec rules each section satisfies.1324. If a test fails, paste the full traceback before attempting a fix.1335. Never fix a bug by relaxing a spec constraint.134 135## KNOWN BUGS TO FIX (from audit)1361. escalation_budget underflow: when budget_violated=True, do not decrement137   escalation_budget. Add guard before the decrement line.1382. hard mode ai_confidence_score: remove the 70/30 probability split.139   Use pure inversion: score = 1.0 - base, always.