HarshDeep61034/guard-rail-rl

GuardianRL Trust & Safety Suite

An interactive, sequential decision-making benchmark designed to train and evaluate AI systems handling digital policy enforcement. Built upon the robust GuardianRL foundation, this framework challenges evaluators (agents) to analyze obscured digital submissions. Evaluators must fetch missing context, classify offenses based on rigid rulebooks, and confidently issue verdicts across 24 unique instances heavily influenced by regional laws (covering the US, EU, India, and the UK).

Live Demonstration: https://huggingface.co/spaces/harsh/guardianrl-trust-safety

The Significance of Digital Policy Enforcement

Maintaining a safe digital sandbox is arguably the most critical operational challenge for modern platforms. Human reviewers process colossal volumes of content while navigating shifting legal guidelines and fragmented context. Their verdicts—whether to allow, flag, remove, or escalate—carry profound implications.

We built this suite to codify and automate this highly complex workflow:

• —Submissions are initially presented with sparse metadata (such as geographic origin, basic engagement stats, and report volume).
• —Evaluators are forced to seek contextual clues actively before arriving at a conclusion. They might need a user's prior strike history or structural thread data.
• —Verdicts vary drastically depending on the regional jurisdiction; speech protected under the US First Amendment might violate the EU's DSA.
• —The scoring mechanism heavily penalizes dual-class failures: both censoring benign conversation and allowing severe toxicity or dangerous advice to propagate.

Because evaluators must execute a sequence of investigative queries before reaching a conclusion, the suite perfectly aligns with RL training paradigms like GRPO and PPO. The final metric is the culmination of a well-executed trajectory, not a single static prediction.

Core GuardianRL Interfaces

The suite fully complies with the GuardianRL REST and WebSocket standards.

REST Endpoints

The WebSocket Gateway

WS /ws

Maintains low-latency, persistent connections essential for multi-stage workflows. The official Python adapter leverages this by default.

Client Transmissions:

{"type": "reset", "data": {"task_id": "basic_toxicity", "seed": 42}}
{"type": "step",  "data": {"action_type": "fetch_user_history", "parameters": {}}}
{"type": "state"}
{"type": "close"}

Server Broadcasts:

{"type": "observation", "data": {"observation": {...}, "reward": 0.20, "done": false}}
{"type": "state",       "data": {...}}
{"type": "error",       "data": {"message": "...", "code": "SESSION_ERROR"}}

Data Topologies

Session Initiation (`/reset`)

// Incoming
{"task_id": "basic_toxicity", "seed": 42}

// Outgoing
{
  "observation": { "post_id": "...", "content": "...", "done": false, ... },
  "reward": null,
  "done": false
}

Workflow Action (`/step`)

// Incoming
{"action": {"action_type": "fetch_user_history", "parameters": {}}}

// Outgoing
{
  "observation": { "step": 1, "user_history": ["..."], "done": false, ... },
  "reward": 0.20,
  "done": false
}

The Observation Manifest

After every maneuver, the environment yields updated situational awareness:

Available Operations

Context Retrieval (Grants +0.20 feedback)

Labeling (Mandatory pre-requisite for termination)

Final Decisions (Concludes the sequence)

Featured Scenarios

basic_toxicity — Default seed: 42 (6 moves max)

A straightforward case of hostility: direct intimidation or dangerous doxxing regardless of region. The context is painfully obvious. Effective evaluators process this after merely an operation or two. Appropriate conclusion: remove

moderate_gray_area — Default seed: 100 (8 moves max)

Nuanced corporate friction: harsh user reviews or unverified corporate leaks. Proper handling hinges heavily on parsing threaded contexts and contrasting them against intricate clauses. EU regulations yield considerably sterner outcomes than standard US protocol here. Appropriate conclusion: allow, flag, or escalate (context-dependent)

advanced_fake_news — Default seed: 777 (10 moves max)

Dangerous biological or health fabrications masking as facts. The criticality is influenced by its virality, current global crises, and the presence of partial truths. Cutting-edge frameworks reliably struggle here, either systematically suppressing harmless speculation or enabling lethal advice to circulate. Appropriate conclusion: remove or flag

Feedback Mechanics

Evaluators are guided via continuous, dense scalar signals.

The extreme penalty for severe negligence mathematically encodes platform asymmetry: ignoring a massive threat generates exponentially more real-world harm than a heavy-handed removal.

Performance Auditing (0.0 – 1.0)

A session's terminal score aggregates four key dimensions:

Because distinct situations demand unique investigative depths, the final grade judges the evaluator linearly against the specific topology of the puzzle, disregarding difficulty tier aggregates.

Interfacing via Python

Download and install the native pipeline tooling:

pip install git+https://huggingface.co/spaces/harsh/guardianrl-trust-safety

from client import SafetyEnv, SafetyAction
import asyncio

async def pipeline():
    async with SafetyEnv(base_url="https://harsh-guardianrl-trust-safety.hf.space") as env:
        session = await env.reset(task_id="basic_toxicity", seed=42)
        state = session.observation

        print(state.content)    # "I know where you live..."
        print(state.geo)        # "EU"

        # Expand context
        session = await env.step(SafetyAction(action_type="fetch_user_history"))
        print(state.user_history)
        print(session.reward)  # 0.20

        # Categorize
        session = await env.step(SafetyAction(
            action_type="mark_violation_type",
            parameters={"violation_type": "harassment"}
        ))

        # Finalize
        session = await env.step(SafetyAction(action_type="remove"))
        print(session.done)    # True
        print(session.reward)  # 0.95

asyncio.run(pipeline())

Generative RL with TRL (GRPO)

Integrate this suite effortlessly within TRL workflows.

from client import SafetyEnv, SafetyAction
from trl.experimental.openenv import generate_rollout_completions

engine = SafetyEnv(base_url="https://harsh-guardianrl-trust-safety.hf.space").sync()
engine.connect()

# Implementation of rollout logic matches standard OpenEnv design patterns. We leverage synchronous environments
# to process token completions mapped to exact GuardianRL actions.

Rapid Diagnostics

Execute the hybrid deterministic/LLM script locally via inference.py.

export HF_TOKEN="your_key_here"
export API_BASE_URL="https://router.huggingface.co/v1"
export MODEL_NAME="meta-llama/Llama-3.3-70B-Instruct"

python inference.py

Output format typically conforms to:

[START] task=basic_toxicity env=trust-safety model=meta-llama/Llama-3.3-70B-Instruct
[STEP] step=1 action=fetch_user_history reward=0.20 done=false error=null
[END] success=true steps=3 score=0.925 rewards=0.20,0.50,0.95

Infrastructure Initialization

git clone https://github.com/harsh/GuardianRL-Trust-Safety-Suite.git
cd GuardianRL-Trust-Safety-Suite

uv sync
cp .env.example .env

uvicorn api.app:app --reload --port 7860

Deploying via containers:

docker build -t guardianrl-safety .
docker run -p 7860:7860 \
  -e HF_TOKEN="your_key" \
  -e MODEL_NAME="your_model" \
  guardianrl-safety