Grish2114/email-prioritization-env

Email Prioritization & Response — OpenEnv Environment

Overview

This is a standard OpenEnv environment simulating corporate email triage. AI agents trained in this environment must learn to classify the urgency of incoming emails, route them to the correct internal department, and draft appropriate, professional replies.

This environment strictly adheres to the OpenEnv specification, providing deterministic step(), reset(), and state() API functions via a FastAPI backend, designed for seamless containerized deployment on Hugging Face Spaces.

Real-World Motivation

Every modern enterprise processes hundreds to thousands of emails daily from customers, partners, and internal staff. Misclassifying an urgent production outage or dropping a critical legal notice causes immense financial and reputational damage. This environment challenges AI agents with a highly realistic, measurable simulation of this exact problem, allowing researchers to evaluate an agent's reasoning, routing, and drafting capabilities safely.

Environment at a Glance

Tasks

Reward Function

Rewards are calculated using a cumulative partial credit system designed to naturally guide policy learning.

• —Task 1: score = priority_score / 0.4 (normalized)
• —Task 2: score = (priority_score + dept_score) / 0.7 (normalized)
• —Task 3: score = priority_score + dept_score + reply_score (direct sum, max 1.0)

Penalties (applied directly to the raw priority score): | Mistake | Penalty | |---------|---------| | Urgent classified as Low | -0.2 | | Low classified as Urgent | -0.1 | | Off-by-one priority | -0.05 |

Action Space

Observation Space

API Endpoints

Quick Start

Local

git clone <repo-url>
cd email-env
pip install -r requirements.txt
uvicorn api.server:app --host 0.0.0.0 --port 7860

Docker

docker build -t email-env .
docker run -p 7860:7860 email-env

Test the API

# Health check
curl http://localhost:7860/health

# List tasks
curl http://localhost:7860/tasks

# Start episode
curl -X POST http://localhost:7860/reset \
  -H "Content-Type: application/json" \
  -d '{"task_id": 1, "seed": 42}'

# Take a step
curl -X POST http://localhost:7860/step \
  -H "Content-Type: application/json" \
  -d '{"task_id": 1, "priority": "urgent", "department": "technical", "reply": ""}'

# Run baseline agent
curl -X POST http://localhost:7860/baseline \
  -H "Content-Type: application/json" \
  -d '{"api_key": "sk-ant-...", "tasks": [1,2,3], "seed": 42}'

Run Baseline Script

export ANTHROPIC_API_KEY=sk-ant-...
python baseline/inference.py

Baseline Scores

Project Structure

• —api/server.py — FastAPI application providing standard endpoints.
• —env/environment.py — Core OpenEnv logic (reset, step, state).
• —env/graders.py — Deterministic reward mechanisms.
• —env/models.py — Pydantic schemas enforcing input validation.
• —env/tasks.py — Metadata describing the 3 difficulty tasks.
• —env/data/emails.json — Evaluative corpus of 30 synthetic emails.
• —baseline/inference.py — Python script executing the autonomous agent.
• —tests/test_environment.py — 12 robust Pytest unit tests.
• —openenv.yaml — Complete metadata specification file.
• —main.py — Standard application entry point.
• —Dockerfile — Configuration for direct Hugging Face Spaces deployment.

OpenEnv Spec Compliance

• —[x] Real-world task simulation (corporate email triage)
• —[x] Typed Pydantic models (Observation, Action, Reward)
• —[x] step() / reset() / state() implemented
• —[x] openenv.yaml with full metadata
• —[x] 3 tasks with difficulty progression (easy/medium/hard)
• —[x] Deterministic graders scoring 0.0-1.0
• —[x] Partial credit reward function
• —[x] Baseline inference script (Claude agent)
• —[x] /baseline /grader /tasks endpoints
• —[x] Dockerfile (HF Spaces compatible)
• —[x] Deployed to Hugging Face Spaces