sumit01-sj/Email_Triage_OpenEnv

📧 Email Triage OpenEnv

A production-grade reinforcement learning environment simulating real-world email triage and management — the kind of work that takes hours of human attention every day in every company.

  

🎯 Motivation

Email overload is a universal workplace problem. Professionals spend 28% of their workday managing email. Effective triage — classifying urgency, routing to the right team, detecting sensitive information, and crafting appropriate responses — requires both domain knowledge and nuanced judgment.

This environment challenges agents to perform this full pipeline, providing rich incremental feedback and progressively harder tasks.

🏗️ Environment Overview

📋 Action Space

Actions are JSON objects with the following types:

Urgency levels: low | medium | high | critical

Categories: billing | support | inquiry | complaint | spam | hr | legal | internal

Departments: billing | technical-support | sales | hr | legal | general

👁️ Observation Space

Each observation is a JSON object containing:

{
  "inbox": [...],
  "processed": [...],
  "step": 3,
  "task_id": "email-triage-hard",
  "instructions": "...",
  "available_actions": ["classify", "route", "flag_pii", "draft_response", "submit_report", "skip"],
  "progress": 0.4,
  "errors": []
}

📌 Tasks

Task 1: Email Classification (Easy)

• —ID: email-triage-easy
• —Emails: 5 realistic business emails
• —Objective: Classify each email by urgency and category
• —Max Steps: 10
• —Score Threshold: 0.75

Task 2: Email Routing + PII Detection (Medium)

• —ID: email-triage-medium
• —Emails: 8 emails (HR, legal, enterprise sales scenarios)
• —Objective: Classify + route to correct department + flag any PII
• —Max Steps: 15
• —Score Threshold: 0.70

Task 3: Full Email Triage Pipeline (Hard)

• —ID: email-triage-hard
• —Emails: 10 emails including a CEO emergency and a regulatory compliance notice
• —Objective: Full pipeline: classify → route → PII → draft critical responses → submit summary report
• —Max Steps: 20
• —Score Threshold: 0.65

🏆 Reward Function

Rewards are incremental — given after every action, not just at episode end.

📊 Baseline Performance

🚀 Setup & Usage

Local Setup

git clone <your-repo>
cd email-triage-openenv
pip install -r requirements.txt
python server.py

Running Inference

export HF_TOKEN="your-api-key"
export API_BASE_URL="https://api.openai.com/v1"
export MODEL_NAME="gpt-4.1-mini"
export TASK_ID="email-triage-hard"

python inference.py

Docker

docker build -t email-triage-openenv .
docker run -p 7860:7860 email-triage-openenv

🔌 API Reference

🧪 Running Tests

python -m pytest tests/ -v

40 tests covering: reset, all three tasks, grader accuracy, reward correctness, and edge cases.

📁 Project Structure

.
├── inference.py          # Hackathon submission entry point
├── server.py             # FastAPI OpenEnv HTTP server
├── openenv.yaml          # Environment metadata
├── requirements.txt
├── Dockerfile
├── README.md
├── env/
│   ├── __init__.py
│   ├── environment.py    # EmailTriageEnv (main class)
│   ├── models.py         # Pydantic models
│   ├── email_data.py     # Realistic email datasets + ground truth
│   └── graders.py        # Deterministic scoring functions
└── tests/
    └── test_environment.py