LC1105/email-triage-env

📧 Email Triage Environment

An OpenEnv environment that simulates real-world email triage — one of the most common productivity tasks in modern workplaces. Agents must classify spam, prioritize inbox urgency, and draft professional replies.

Environment Description

Email triage is something every professional does daily. This environment provides a structured, gradable simulation with three tasks of increasing difficulty, making it ideal for training and evaluating LLM-based agents on practical workplace skills.

Tasks

Task 1: spam_classification — Easy

The agent is shown 5 emails one at a time and must classify each as spam or not_spam.

• —Steps: 5 (one per email)
• —Action: Single word — spam or not_spam
• —Grader: Exact match against ground truth labels
• —Score per step: 1.0 (correct) or 0.0 (incorrect)
• —Episode score: Average across all 5 emails

Task 2: priority_ranking — Medium

The agent sees 5 workplace emails simultaneously and must rank them from highest to lowest urgency.

• —Steps: 1
• —Action: Comma-separated email IDs e.g. 1, 3, 5, 4, 2
• —Grader: Normalized Kendall tau distance (measures how many pairs are correctly ordered)
• —Score: 0.0–1.0 with partial credit for partially correct orderings

Task 3: reply_drafting — Hard

The agent reads a complex customer complaint email and must draft a complete professional reply.

• —Steps: 1
• —Action: Full email reply text
• —Grader: Deterministic keyword matching across 5 required elements:
• —Acknowledges specific order/issue (20%)
• —Genuine apology/empathy (20%)
• —Concrete resolution offer (25%)
• —Timeline commitment (20%)
• —Direct contact info (15%)
• —Score: Weighted sum, with penalty for replies < 50 words

Action & Observation Spaces

Action

class EmailTriageAction(Action):
    response: str   # The agent's triage decision or reply text

Observation

class EmailTriageObservation(Observation):
    task_name: str      # Active task identifier
    prompt: str         # Instructions for this step
    email_content: str  # Email(s) to process
    feedback: str       # Feedback from last action
    step_score: float   # Reward for last step [0.0, 1.0]
    done: bool          # Episode complete flag
    reward: float       # Reward signal

API Endpoints

Setup & Usage

Install locally

pip install openenv-core
pip install -e .

Run the server

EMAIL_TRIAGE_TASK=spam_classification uvicorn email_triage_env.server.app:app --host 0.0.0.0 --port 7860

Run with Docker

docker build -t email-triage-env .
docker run -p 7860:7860 -e EMAIL_TRIAGE_TASK=spam_classification email-triage-env

Run inference baseline

export HF_TOKEN=your_hf_token_here
export API_BASE_URL=https://router.huggingface.co/v1
export MODEL_NAME=Qwen/Qwen2.5-72B-Instruct

python inference.py

Quick Python example

import os
from email_triage_env.models import EmailTriageAction
from email_triage_env.server.email_triage_env import EmailTriageEnvironment

os.environ["EMAIL_TRIAGE_TASK"] = "spam_classification"
env = EmailTriageEnvironment()

obs = env.reset()
print(obs.prompt)
print(obs.email_content)

action = EmailTriageAction(response="spam")
obs = env.step(action)
print(obs.feedback)   # "CORRECT! ..."
print(obs.reward)     # 1.0

Baseline Scores

Environment Variables

*Required only for running inference.py

Reward Design

The reward function provides dense, meaningful signal:

• —spam_classification: Binary per-step reward (0 or 1), averaged across 5 steps. Encourages consistent classification accuracy.
• —priority_ranking: Graded partial credit using Kendall tau correlation. Rewards any improvement in ordering — not just perfect rankings.
• —reply_drafting: Multi-element weighted scoring. Each key element of a good customer reply contributes independently, with a length penalty preventing trivially short responses.

Validation

pip install openenv-core
openenv validate

Disqualification Checklist ✅

• —[x] HF Space deploys and responds to /reset
• —[x] openenv validate passes
• —[x] docker build succeeds
• —[x] inference.py runs and produces structured logs
• —[x] 3+ tasks with graders producing scores in [0.0, 1.0]