Kshtitij/ticket-triage
Customer Support Ticket Triage Environment
An OpenEnv-compliant reinforcement-learning environment where an AI agent processes customer support tickets by classifying issues, assigning priorities, and generating appropriate customer responses.
Overview
Each episode contains 10 realistic support tickets drawn from four categories (billing, technical, account, general). The agent processes tickets one at a time; after all tickets are handled the episode ends (done=True).
The environment is fully deterministic – the same ticket corpus is used every run, ensuring reproducible scores across models.
Project Structure
project/
├── env.py # Core environment (reset / step / state)
├── models.py # Pydantic models: Observation, Action, Reward
├── tasks/
│ ├── easy.py # Task 1 – category classification only
│ ├── medium.py # Task 2 – classification + priority
│ └── hard.py # Task 3 – classification + priority + response
├── inference.py # Baseline inference script (OpenAI client)
├── openenv.yaml # Environment & task metadata
├── Dockerfile # Container definition
├── requirements.txt # Python dependencies
└── README.mdObservation Space
Action Space
Reward Function
Rewards are step-wise (returned after each ticket), clamped to [0.0, 1.0].
Tasks
Task 1 – Easy (Category Only)
- Reward:
1.0 × category_score - Goal: Correctly classify the ticket into one of the four categories.
- Expected baseline score: ~0.85
Task 2 – Medium (Category + Priority)
- Reward:
0.5 × category_score + 0.5 × priority_score - Goal: Correctly classify and assign the appropriate urgency level.
- Expected baseline score: ~0.75
Task 3 – Hard (Category + Priority + Response)
- Reward:
0.3 × category_score + 0.3 × priority_score + 0.4 × response_score - Goal: Full triage: classify, prioritize, and write a helpful response.
- Expected baseline score: ~0.65
Setup
Local (Python)
# Clone / enter project directory
cd project/
# Install dependencies
pip install -r requirements.txt
# Set required environment variables
export OPENAI_API_KEY="sk-..."
export MODEL_NAME="gpt-4o-mini" # optional, default: gpt-4o-mini
export API_BASE_URL="https://api.openai.com/v1" # optionalDocker
docker build -t ticket-triage .
docker run --rm \
-e OPENAI_API_KEY="sk-..." \
-e MODEL_NAME="gpt-4o-mini" \
ticket-triageRunning Inference
# Run all 3 tasks and print scores
python inference.pyOutput follows this strict log format:
[START] task=easy model=gpt-4o-mini base_url=https://api.openai.com/v1
[STEP] task=easy step=00 ticket=T001 category=billing priority=low reward=1.0000 ...
...
[END] task=easy avg_reward=0.9000 steps=10 elapsed=12.34s
[START] task=medium ...
...
[END] task=hard avg_reward=0.6500 steps=10 elapsed=18.22s
======================================================================
FINAL RESULTS
======================================================================
Task [easy ] avg_reward=0.9000 steps=10
Task [medium] avg_reward=0.7500 steps=10
Task [hard ] avg_reward=0.6500 steps=10
AGGREGATE SCORE : 0.7667Results are also saved to `results.json`.
Using the Environment Directly
from env import TicketTriageEnv
from models import Action
env = TicketTriageEnv(task_mode="hard")
obs = env.reset()
while obs is not None:
action = Action(
category="billing",
priority="high",
response="We apologize for the duplicate charge. A full refund will be processed within 3-5 business days."
)
obs, reward, done, info = env.step(action)
print(f"Reward: {reward.total:.4f} Done: {done}")
print("Final state:", env.state())Expected Baseline Scores
Scores are deterministic for a given model at `temperature=0`.
Constraints
- Runs within 2 vCPU / 8 GB RAM
- No external APIs beyond OpenAI
- No randomness – fully reproducible