harshitworkgupta/Privacy-Redaction-OpenEnv

AI Privacy Redaction & Data Sharing Environment

A fully containerized OpenEnv compliant environment designed to simulate real-world data cleaning and privacy mitigation tasks.

Motivation

Redacting PII (Personal Identifiable Information) is a critical compliance and security task across the globe (GDPR, HIPAA). This environment provides a sandbox to train and benchmark autonomous agents tasked with scrubbing data efficiently without negatively impacting readability through over-redaction.

Environment Spec (OpenEnv Compliant)

Observation Space

The environment emits a typed Observation schema defining the current document snapshot:

{
  "document": "string (full original text)",
  "tokens": ["list", "of", "strings"],
  "current_index": "integer (which token is currently being scanned)",
  "policy": {
    "redact": ["phone", "email"],
    "preserve": ["names"]
  }
}

Action Space

The environment expects a typed Action schema modifying the token at current_index:

{
  "type": "redact" | "keep" | "replace",
  "token_index": "integer",
  "replacement": "string | null (used if type is 'replace')"
}

Reward Function

The deterministic grader normalizes scores dynamically (0.0 to 1.0) tracking step logic:

• —+1.0 For correctly matching a redacted sensitive token.
• —-1.0 For missing a sensitive token.
• —-0.5 Over-reduction (stripping safe textual utility).
• —+0.2 Rewarded for maintaining readability safely.

Once an episode completes, a final_score (from 0.0 - 1.0) is returned detailing the final benchmark efficiency.

Tasks Roadmap

The openenv.yaml governs 3 core benchmarks:

• —Easy: Standard clear-text phone numbers and simple emails.
• —Medium: Medical datasets featuring IDs, mixed name patterns, and mixed capitalization.
• —Hard: Heavy token obfuscation contexts (e.g. john at gmail dot com, five five five zero).

Usage & Deployment

Run Containerized

This system natively hosts both its API and the React GUI on port 7860, perfect for HuggingFace Spaces.

docker build -t privacy_env .
docker run -p 7860:7860 privacy_env

Open http://localhost:7860 in your web browser.

Run Evaluator Baseline

Included is baseline/run_agent.py, capable of hitting models like gpt-4o-mini to baseline all 3 tasks programmatically.

export OPENAI_API_KEY="sk-..."
python baseline/run_agent.py