ItsMe1501/MetaHackathon

Email Triage OpenEnv Environment

Description & Motivation

The Email Triage environment simulates a real-world cognitive task: processing and organizing a busy inbox. AI agents are often tasked with managing emails—from handling routine support requests and meeting declines to eliminating spam and archiving newsletters. This environment provides a rigorous testbed to verify whether an agent can correctly identify an email's intent and select the appropriate action while avoiding destructive behaviors on important emails.

State Space Definitions

Action Space (EmailTriageAction)

• —command (Literal): The action to perform. One of: read, reply, forward, archive, delete, flag, or done.
• —email_id (Optional[str]): The ID of the email to target (required for all actions except done).
• —target (Optional[str]): The target email address (for forward) or message content (for reply).

Observation Space (EmailTriageObservation)

• —inbox_summary (List[EmailSummary]): A list showing the current status of all emails in the inbox (e.g., whether they are archived, deleted, flagged, etc.).
• —current_email (Optional[Email]): Displays the full body of the recently read email when the read command is used.
• —feedback (str): Textual feedback indicating the result of the last action or identifying errors.

Task Descriptions & Difficulty

The environment cycles through three built-in tasks of increasing difficulty:

1. 1.Task "easy":
2. 2.Objective: Delete all spam emails while leaving legitimate emails in the inbox.
3. 3.Difficulty: Easy. Agents only need to perform a simple classification boundary and issue single commands (delete).

1. 1.Task "medium":
2. 2.Objective: Archive newsletters, flag emails from a VIP (boss@comp.com), and delete spam.
3. 3.Difficulty: Medium. Requires distinguishing between three distinct categories of emails and executing correct corresponding actions without disturbing normal "ham" emails.

1. 1.Task "hard":
2. 2.Objective: Reply "Decline" to meeting requests, forward support tickets specifically to support@comp.com, and archive resolved system alerts.
3. 3.Difficulty: Hard. Agents must extract context from the email body, formulate specific payloads in their actions (e.g., target email addresses or reply strings), and correctly map complex intents to actions.

Reward Function

The environment provides a potential-based dense tracking return:

• —Rewards are computed incrementally after every single action.
• —The environment computes a "completion score" (0.0 to 1.0) based on the proportion of emails in their target terminal states.
• —By providing the delta of this score after each step, the agent receives partial progress rewards and active penalties if it incorrectly modifies an already categorized email.
• —Terminating the episode with done issues the final delta completion reward. A maximum step cutoff inherently penalizes infinite loops.

Setup and Usage Instructions

Run the FastAPI Server

1. 1.Ensure you have uv installed (pip install uv).
2. 2.Run uv lock to generate the lockfile.
3. 3.Install dependencies: uv sync.
4. 4.Run the API locally: uvicorn server.app:app --host 0.0.0.0 --port 8000.

Containerized Execution

You can easily build and test the environment in a Docker container using the provided server/Dockerfile.

docker build -t openenv-email-triage -f server/Dockerfile .
docker run -p 8000:8000 openenv-email-triage

Note: This Space should be tagged with `openenv` when deployed to Hugging Face Spaces using the standard FastAPI Docker Space template.

Baseline Inference

A baseline inference script (baseline.py) is provided that uses GPT-4o to iterate through all three tasks and evaluate its performance.

export OPENAI_API_KEY="sk-..."
python baseline.py

Expected Baseline Scores

Because this is an LLM agent natively understanding context, GPT-4o typically scores perfect on this environment:

• —Task easy: 1.0
• —Task medium: 1.0
• —Task hard: 1.0
• —Average: 1.00