bandanagupta-03/opd-queue-env

🏥 Healthcare OPD Queue Optimization Environment

OpenEnv Hackathon Submission · Real-world hospital simulation · 3 tasks · RL-ready

   

🎯 Project Overview

The OPD Queue Optimization Environment simulates a hospital Outpatient Department under realistic operating conditions. An AI agent must intelligently assign arriving patients to available doctors, balancing:

• —Urgency — critical patients must be seen quickly
• —Fairness — lower-severity patients shouldn't wait indefinitely
• —Efficiency — doctors must not sit idle when patients are waiting
• —Resilience — surge events (accidents, mass casualties) must be handled gracefully

This environment was designed to benchmark AI planning agents in healthcare resource allocation — a domain where good decisions have direct human impact.

🌍 Real-World Motivation

Every day, millions of patients visit hospital OPDs worldwide. Inefficient queue management leads to:

• —Preventable deterioration of critical patients
• —Long wait times causing patient distress
• —Doctor burnout from uneven workload distribution
• —Wasted medical capacity during off-peak hours

AI agents trained in this environment can be adapted to assist real triage nurses and queue managers, augmenting human decision-making with data-driven recommendations.

🏗️ Architecture

opd_queue_env/
│
├── envs/opd_env/
│   ├── __init__.py        ← Public API exports
│   ├── env.py             ← Core OPDQueueEnv (reset, step, state)
│   └── models.py          ← Pydantic models: Observation, Action, Reward
│
├── tasks/
│   └── __init__.py        ← Task definitions + deterministic graders
│
├── scripts/
│   ├── run_baseline.py    ← Rule-based baseline agent
│   └── train_rl_agent.py  ← Q-Learning training + evaluation
│
├── inference.py           ← OpenAI-driven inference (mandatory format)
├── app.py                 ← FastAPI server (POST /reset, POST /step)
├── openenv.yaml           ← OpenEnv metadata spec
├── Dockerfile             ← Docker build (python:3.9-slim, port 7860)
├── requirements.txt       ← Python dependencies
└── README.md              ← This file

Runtime Flow

Patient Arrives ──► Queue ──► Agent Decision ──► Doctor Assignment
       ↑                           │
       │                           ▼
  Surge Event           Reward Computation
       │                  (6 sub-components)
       │                           │
  Escalation ◄─── Long Wait        ▼
                            Next Observation

🔍 Observation Space

Patient fields: patient_id, severity (1-5), waiting_time, arrival_time, is_being_seen, escalation_count

Doctor fields: doctor_id, is_available, current_patient, patients_seen_today, fatigue_factor (1.0+), busy_until

⚡ Action Space

{
  "patient_id": "P0012",        // Which patient to treat (null = idle)
  "doctor_id": "D01",           // Which doctor to assign (null = auto-pick)
  "priority_override": false    // Emergency override for severity-5 cases
}

Discrete action strategies (used by RL agent):

• —0 — Idle (no assignment this step)
• —1 — Assign highest-severity patient
• —2 — Assign longest-waiting patient
• —3 — Assign by composite score (severity × 10 + wait × 0.5)

🏆 Reward Function

The reward is continuous, decomposed, and informative at every step:

Total reward = sum of all components. Ranges approximately from -5.0 to +5.0 per step.

🌟 Innovative Features

1. Emergency Escalation System

If a patient waits beyond the task-specific threshold (10-20 minutes), their severity automatically increases by 1. This creates urgency and directly penalises agents that ignore mild patients too long.

2. Doctor Fatigue Simulation

Each doctor's fatigue_factor grows logarithmically with patients treated:

fatigue_factor = 1.0 + 0.05 × min(patients_seen, 15)

This multiplies treatment duration, simulating real physician cognitive and physical fatigue.

3. Dynamic Surge Events

With task-specific probability, a mass-casualty event spawns 2-8 high-severity patients in a single timestep — simulating road accidents, building collapses, or disease outbreaks. The surge_active flag warns the agent.

4. Smart Reward Shaping

The six-component decomposed reward allows RL agents to learn which behaviours are beneficial vs harmful, rather than from a single sparse signal.

📋 Task Descriptions

All tasks use the same API; task selection at env.reset(task_id=...).

🚀 Setup Instructions

Local Installation

# Clone or unzip the project
cd opd_queue_env

# Install dependencies
pip install -r requirements.txt

# Run FastAPI server
uvicorn app:app --host 0.0.0.0 --port 7860 --reload

# Open in browser:
# http://localhost:7860/docs

Run Baseline Agent

python scripts/run_baseline.py

Train RL Agent

# Train on one task
python scripts/train_rl_agent.py --task easy --episodes 300

# Train on all tasks
python scripts/train_rl_agent.py --task all --episodes 500

Run Inference

# With LLM (requires OPENAI_API_KEY)
export OPENAI_API_KEY=sk-...
python inference.py --task medium --steps 30 --model gpt-4o-mini

# Without LLM (rule-based fallback)
python inference.py --task hard --steps 30 --no-llm

Docker

docker build -t opd-queue-env .
docker run -p 7860:7860 opd-queue-env

📊 Baseline Results

Rule-based agent scores (seed=42, reproducible):

Q-Learning agent (300 episodes training, seed=42):

🔌 API Quick Reference

# Reset environment
curl -X POST http://localhost:7860/reset \
  -H "Content-Type: application/json" \
  -d '{"task_id": "medium", "seed": 42}'

# Take a step
curl -X POST http://localhost:7860/step \
  -H "Content-Type: application/json" \
  -d '{"patient_id": "P0001", "doctor_id": "D01", "priority_override": false}'

# Get current state
curl http://localhost:7860/state

# Get graded score
curl -X POST http://localhost:7860/score

📄 OpenEnv Compliance

This environment implements the full OpenEnv interface:

• —✅ Typed Pydantic models: Observation, Action, Reward
• —✅ Core API: reset(), step(), state()
• —✅ step() returns (observation, reward, done, info)
• —✅ openenv.yaml metadata file
• —✅ 3 tasks with deterministic graders (score 0.0 → 1.0)
• —✅ Continuous, informative reward function
• —✅ Baseline agent + RL agent scripts
• —✅ Mandatory inference.py with [START]/[STEP]/[END] format
• —✅ FastAPI server with POST /reset and POST /step
• —✅ Docker support (python:3.9, port 7860)
• —✅ HuggingFace Spaces deployment ready

📝 License

MIT License — free to use, modify, and distribute.