sc-likes-to-code/openenv-customer-support-env

🎫 Support Ticket Resolution Environment (OpenEnv)

A real-world, multi-step customer support simulation environment built on the OpenEnv framework. Designed to train and evaluate AI agents on tasks that mirror genuine human support workflows.

🌍 Overview

Unlike toy RL environments, this system simulates a realistic customer support pipeline where an agent must:

• —Classify support tickets by issue type
• —Detect sentiment and apply policy-aware responses
• —Ask clarifying questions when information is missing
• —Use conversation memory across turns
• —Decide whether to resolve or escalate issues

This makes it a high-value benchmark for evaluating multi-step reasoning, policy compliance, and stateful decision-making in AI agents.

🏗️ Project Structure

my-env/
├── inference.py          # Baseline agent — runs all 3 tasks
├── models.py             # Pydantic models: Action, Observation, Reward
├── openenv.yaml          # OpenEnv spec metadata (name, tasks, spaces, rewards)
├── pyproject.toml        # Project metadata and dependencies
├── requirements.txt      # Python dependencies
├── uv.lock               # Locked dependency versions
├── Dockerfile            # Container definition
├── client.py             # HTTP client for the environment
├── __init__.py           # Root package
└── server/
    ├── __init__.py       # Server package
    ├── app.py            # FastAPI server (reset / step / state / health)
    ├── grader.py         # Task graders with reward shaping
    ├── tasks.py          # Task definitions (easy / medium / hard)
    └── your_environment.py  # Core SupportEnv class

📜 OpenEnv Spec (openenv.yaml)

This environment is fully compliant with the OpenEnv specification:

name: openenv-customer-support-env
version: "1.0.0"
tags: [openenv]
entrypoint: server.app:app
tasks:
  - id: easy   | difficulty: easy   | max_steps: 6
  - id: medium | difficulty: medium | max_steps: 6
  - id: hard   | difficulty: hard   | max_steps: 8

Validated via:

openenv validate

⚙️ Core API

reset(task: str) -> Observation        # Initialize episode for given task
step(action: Action) -> (Observation, Reward, done, info)  # Take one action
state() -> dict                        # Inspect full current episode state

HTTP Endpoints

📊 Observation Space

{
  "tickets": [
    {"id": 1, "text": "Customer message here"}
  ],
  "current_ticket_id": 1,
  "history": [
    {"user": "...", "agent": "...", "action_type": "classify"}
  ]
}

🎮 Action Space

Action(
    action_type: str,       # "classify" | "respond" | "escalate" | "ask"
    ticket_id: int,         # ID of the ticket being handled
    content: Optional[str]  # Classification label or response text
)

📋 Tasks

🟢 Easy — Ticket Classification & Response

Max steps: 6

Agent must classify the ticket and provide an appropriate response.

Max achievable: 1.0

🟡 Medium — Sentiment-Aware Policy Resolution

Max steps: 6

Agent must detect issue type, show empathy, and generate a policy-compliant response.

Max achievable: 1.0

🔴 Hard — Multi-Turn Memory-Based Resolution

Max steps: 8

Agent must follow the full sequence: classify → ask → respond, using conversation memory.

Penalties:

Max achievable: 1.0

🏆 Reward Design

This environment uses dense reward shaping — agents receive meaningful signal at every step, not just at episode end.

• —Partial credit for each correct intermediate action
• —Efficiency bonuses for faster resolution
• —Memory bonuses for using context from prior turns
• —Penalties for skipping required steps, repeating actions, or escalating unnecessarily
• —Episode ends early on near-perfect score (≥0.95) or after max steps

📈 Baseline Scores

Achieved by the fallback rule-based agent (no LLM, no API key required):

A frontier LLM agent is expected to score significantly higher.

🚀 Setup Instructions

Clone repository

git clone <repo-url>
cd my-env

Create virtual environment

python -m venv venv
venv\Scripts\activate      # Windows
source venv/bin/activate   # Linux/Mac

Install dependencies

pip install -r requirements.txt

🔐 Environment Variables

If no API key is set, the agent runs in fallback mode using rule-based actions. All 3 tasks still complete successfully.

▶️ Run Inference

python inference.py

Runs all 3 tasks in sequence and outputs:

[START] task=easy env=support_env model=Qwen/Qwen2.5-72B-Instruct
[STEP] step=1 action={...} reward=0.50 done=false error=null
[STEP] step=2 action={...} reward=0.50 done=true error=null
[END] success=true steps=2 rewards=0.50,0.50
[SUMMARY] task=easy score=0.500 success=true steps=2
...
[AGGREGATE] tasks=3 avg_score=0.433

🐳 Docker Usage

docker build -t support-env .
docker run -p 7860:7860 support-env

Test endpoints:

curl -X POST http://localhost:7860/reset?task=easy
curl http://localhost:7860/health

✅ OpenEnv Validation

pip install openenv-core
openenv validate

☁️ Deployment

Deployed as a Hugging Face Docker Space — fully containerized, CPU-friendly, and responds within the 20-minute inference runtime limit.

🔮 Future Improvements

• —Multi-ticket queue handling
• —Memory persistence across episodes
• —Advanced policy rule engine
• —Human-in-the-loop simulation
• —More task difficulty levels

📄 License

MIT