siddham-jain/multi-channel-exec-env

Executive Assistant Environment (Multi-Channel Task Orchestrator)

An OpenEnv-compatible reinforcement learning environment where an AI agent acts as a personal executive assistant, managing and coordinating across email, messages (Slack-style channels), and calendar to complete real-world workplace tasks.

  

Motivation

Scheduling meetings, resolving conflicts, and coordinating across email and chat is one of the most common knowledge-worker tasks — millions of people do it daily. This environment provides a controlled, reproducible setting to train and evaluate AI agents on multi-channel coordination, a capability that has immediate real-world value for:

• —Personal assistant products (Google Assistant, Siri, Cortana)
• —Enterprise workflow automation (Slack bots, Outlook add-ins)
• —Autonomous agent frameworks (AutoGPT, CrewAI, LangGraph)

Unlike single-tool environments, this requires the agent to synthesize information across channels and produce verifiable state changes (correct calendar entries, correct emails sent to correct people).

Key Features

• —Multi-Channel Coordination: Synthesis of information across email, Slack channels, and a calendar.
• —YAML-Driven Architecture: openenv.yaml is the single source of truth for task descriptions, initial states, and grading logic.
• —Concurrent Session Support: Class-level session store with unique episode_id for independent, simultaneous agent sessions.
• —Extensible Grading Engine: Deterministic, programmatic grader with partial credit for complex state changes.
• —Full OpenEnv Spec Compliance: Typed models, /reset, /step, /state, /schema, /metadata, /health, /mcp endpoints.

Architecture

exec-assistant-env/
├── __init__.py                     # Package exports
├── models.py                       # Pydantic: AssistantAction, AssistantObservation, AssistantState
├── client.py                       # HTTP client for training/evaluation code
├── inference.py                    # Baseline agent script (OpenAI client)
├── openenv.yaml                    # OpenEnv manifest (tasks, grading, metadata)
├── pyproject.toml                  # Dependencies + [project.scripts] entry point
├── uv.lock                         # Locked dependencies for reproducibility
├── Dockerfile                      # Container image (uv-based)
└── server/
    ├── __init__.py
    ├── app.py                      # create_app (openenv) + custom /state endpoint
    ├── assistant_environment.py    # Environment logic, class-level session management
    └── grader.py                   # Deterministic grading engine

Action Space

The agent can perform 9 action types:

All actions are submitted as a single AssistantAction Pydantic model with action_type and optional fields.

Observation Space

Each step returns an AssistantObservation containing:

Tasks

Task 1: Reply and Schedule (Easy)

Difficulty: Easy · Max Steps: 15 · Objectives: 3

Alice emailed requesting a 30-minute meeting with Bob on July 15 at 2 PM. The agent must read the email, create the calendar event with correct attendees/time, and reply to Alice confirming.

Expected score (frontier model): 0.85–1.0

Task 2: Conflict Resolution Scheduling (Medium)

Difficulty: Medium · Max Steps: 20 · Objectives: 4

Dave messages in #scheduling requesting a 1-hour meeting with Carol and Eve at 10 AM on July 16. Carol already has a conflict at that time. The agent must detect the conflict, reschedule to 11 AM, and notify everyone in the channel.

Expected score (frontier model): 0.60–0.90

Task 3: Multi-Channel Coordination (Hard)

Difficulty: Hard · Max Steps: 25 · Objectives: 7

Frank emails requesting a 1-hour product demo on July 17. The CEO specifies via #executive that Bob and Eve must attend. Both have calendar conflicts earlier in the day, leaving 15:00 as the only free slot. The agent must read all channels, find the correct slot, create the event, email Frank, and notify both #executive and #team.

Expected score (frontier model): 0.40–0.75

Task 4: Executive Rescheduling & Conflict Resolution (Expert)

Difficulty: Expert · Max Steps: 30 · Objectives: 6

A mandatory Board of Directors meeting must happen July 18 at 10 AM (2 hours, Alice + Bob required). A Product Sync is already scheduled at that slot. The agent must reschedule the Product Sync to the earliest free afternoon slot (13:30), notify affected participants, and create the Board meeting.

Expected score (frontier model): 0.20–0.55

Reward Design

The reward function provides continuous signal over the full trajectory:

• —Per-step cost: −0.02 (discourages idle actions / infinite loops)
• —Objective completion: Proportional to objective point value (awarded immediately on detection)
• —Partial credit: Calendar events are scored on date, time, attendees, and duration — not all-or-nothing
• —Invalid action penalty: −0.05 for malformed or unknown actions
• —Done bonus: 50% of final graded score awarded as bonus when agent signals completion

Reward is not sparse — the agent receives signal for each sub-task completed and the score accumulates monotonically.

Grader

Each task has a deterministic, programmatic grader that:

• —Checks completed objectives against a per-task registry
• —Scores each objective 0.0–1.0 based on verifiable state changes (emails sent, events created, messages posted)
• —Does not judge prose quality — only structural correctness
• —Is fully reproducible: same actions → same score
• —Produces different scores for different action sequences (verified: 0.0, 0.15, 0.65, 1.0)

Setup & Usage

Prerequisites

pip install uv        # fast Python package manager

Local Development

git clone <this-repo>
cd exec-assistant-env

# Install dependencies
uv sync

# Start the server
uv run uvicorn server.app:app --host 0.0.0.0 --port 7860

# Test health
curl http://localhost:7860/health
# → {"status": "healthy"}

Docker

docker build -t exec-assistant-env .
docker run -p 7860:7860 exec-assistant-env

curl http://localhost:7860/health

Run Baseline Agent

cp .env.example .env
# Edit .env: set HF_TOKEN, API_BASE_URL, MODEL_NAME

uv run python inference.py
# Prints [START], [STEP]..., [END] logs

Against a live HF Space:

ENV_BASE_URL=https://siddham-jain-multi-channel-exec-env.hf.space \
MY_ENV_V4_TASK=easy \
uv run python inference.py

Interactive Usage (Python)

from client import ExecAssistantEnv
from models import AssistantAction, ActionType

env = ExecAssistantEnv(base_url="http://localhost:7860")

obs = env.reset(task_id="easy")
print(obs.task_description)

result = env.step(AssistantAction(action_type=ActionType.READ_EMAILS))
print(result["observation"].action_result)

result = env.step(AssistantAction(
    action_type=ActionType.CREATE_EVENT,
    event_title="Sync with Bob",
    event_date="2025-07-15",
    event_time="14:00",
    event_duration_minutes=30,
    attendees=["alice", "bob"],
))
print(result["reward"])  # Positive reward for correct event

state = env.state()
print(f"Score: {state.score}, Completed: {state.completed_objectives}")

API Endpoints

Baseline Scores

Using Qwen/Qwen2.5-72B-Instruct via HF Inference Router, temperature=0:

Scores are approximate and may vary slightly by model version and temperature.

OpenEnv Spec Compliance

License

BSD-3-Clause

Deployment on Hugging Face Spaces

This environment is designed to be deployed as a Hugging Face Space using the Docker SDK.

Deployment Summary

• —Space Name: siddham-jain/multi-channel-exec-env
• —SDK: docker
• —Status: Ready for deployment (with web interface enabled)

To deploy or update the space, ensure your README.md contains the correct YAML frontmatter and run your deployment script or use the Hugging Face CLI:

# Example deployment via openenv or git
git push huggingface main