Hr1th1k17/razorpay-itch

Razorpay Itch: Autonomous Fleet & Freight Dispatcher

An OpenEnv / Gymnasium reinforcement learning environment for the logistics pain point:

Why can't manufacturers rely on guaranteed, accountable pickup logistics for outbound deliveries?

This project models outbound pickup logistics as a dynamic vehicle routing and capacity-constrained dispatch problem. An RL agent controls a fleet of trucks that must pick up factory cargo, return it to a hub, and prevent factory loading docks from overflowing.

Why this is a strong hackathon problem

• —It maps to a real operations problem, not a toy environment.
• —It has meaningful partial-progress rewards instead of a binary win/loss signal.
• —It combines routing, capacity management, backlog control, and limited operating time.
• —It is directly trainable with standard RL tooling.

Screenshots

Initial environment state after reset:

[image]

Mid-episode state after several dispatch steps:

[image]

Demo walkthrough

Suggested live demo flow:

1. 1.Introduce the logistics problem in one sentence.
2. 2.Show the environment screenshots and explain factories, trucks, and hub.
3. 3.Run demo.py to show a random rollout.
4. 4.Show train.py and the saved artifacts in artifacts/.
5. 5.Close with the evaluation result and why this is a strong RL benchmark.

Environment design

Observation space

The observation is a fixed-length vector built from:

• —factory backlog, fullness, cargo wait time, and coordinates
• —truck position, remaining capacity, utilization, current status, and target
• —traffic grid features
• —global congestion and utilization signals

Action space

The action space is discrete but semantically meaningful:

• —0: NOOP
• —1..N: smart-dispatch a truck to the best available factory
• —N+1..2N: return a truck to the hub
• —remaining actions: explicit truck-to-factory assignments

The OpenEnv adapter exposes the same logic through typed actions:

• —SMART_DISPATCH
• —DISPATCH_TO_FACTORY
• —RETURN_TO_HUB
• —NOOP

Reward shaping

The reward is designed for partial progress:

• —positive reward for successful pickups
• —higher reward for efficient truck utilization
• —positive reward for returning cargo to the hub
• —penalty when factories remain full
• —penalty for impossible dispatches
• —penalty for ending an episode with undelivered cargo on trucks

Task graders

The OpenEnv observation includes three explicit task scores in the 0.0 to 1.0 range:

• —easy_deliver_one_load: 1.0 once at least one load has been delivered, else 0.0
• —medium_fleet_efficiency: current average fleet utilization
• —hard_zero_overflow: starts at 1.0 and drops as overflow events accumulate

Repo layout

razorpay_itch_env/
  __init__.py
  env.py
  models.py
  openenv_models.py
  openenv_env.py
  spaces.py
  generator.py
  renderer.py
train.py
evaluate.py
demo.py
openenv_app.py
inference.py
openenv.yaml
Dockerfile
tests/
artifacts/
media/

Quickstart

1. Create and activate a virtual environment

python -m venv venv
venv\Scripts\Activate.ps1

2. Install dependencies

pip install -r requirements.txt

Note: pygame is intentionally excluded because it fails to build on Python 3.14 on Windows and is not required for the current matplotlib renderer or PPO baseline.

3. Run a random-policy sanity check

venv\Scripts\python.exe demo.py --episodes 2 --max-steps 25 --seed 42

4. Train a PPO baseline

venv\Scripts\python.exe train.py --timesteps 20000 --n-envs 2 --seed 42

5. Evaluate the saved model

venv\Scripts\python.exe evaluate.py --model-path artifacts\ppo_razorpay_itch.zip --episodes 10 --seed 42

6. Run the OpenEnv server

venv\Scripts\python.exe -m uvicorn openenv_app:app --host 0.0.0.0 --port 8000

To enable the built-in OpenEnv web UI:

$env:ENABLE_WEB_INTERFACE="true"
venv\Scripts\python.exe -m uvicorn openenv_app:app --host 0.0.0.0 --port 8000

Then open:

http://localhost:8000/web

7. Run LLM inference

Set the required environment variables and run the root inference script:

$env:API_BASE_URL="https://your-openai-compatible-endpoint/v1"
$env:MODEL_NAME="your-model-name"
$env:HF_TOKEN="your-token"
venv\Scripts\python.exe inference.py --max-steps 25 --seed 42

inference.py uses the OpenAI Python client and converts model JSON output into FreightAction objects.

Usage guide

Run the demo

Use the demo script to prove the environment resets, steps, and returns metrics correctly.

venv\Scripts\python.exe demo.py

Expected output format:

{"episode": 0, "steps": 25, "total_reward": 7.51, "cargo_delivered": 1, "cargo_picked_up": 23, "overflow_events": 55, "fuel_consumed": 237.89}

Train your own baseline

The included PPO script uses:

• —Monitor for episode logging
• —VecNormalize for observation and reward normalization
• —EvalCallback for checkpointing and evaluation

Example:

venv\Scripts\python.exe train.py --timesteps 100000 --n-envs 4 --seed 42

Outputs are saved to:

• —artifacts/ppo_razorpay_itch.zip
• —artifacts/vecnormalize.pkl
• —artifacts/tensorboard/
• —artifacts/eval_logs/

Evaluate a trained model

venv\Scripts\python.exe evaluate.py --model-path artifacts\ppo_razorpay_itch.zip

Use the OpenEnv adapter

This repo now supports two interfaces over the same logistics simulation:

• —Gymnasium environment for PPO training in razorpay_itch_env/env.py
• —OpenEnv server adapter for typed API access in razorpay_itch_env/openenv_env.py

Typed OpenEnv request and response models are defined in razorpay_itch_env/openenv_models.py.

Deployment files

The repo includes the root deployment files required for OpenEnv/Hugging Face packaging:

• —openenv.yaml
• —Dockerfile
• —inference.py

Baseline results

Random-policy sanity check:

{"episode": 0, "steps": 25, "total_reward": 7.51, "cargo_delivered": 1, "cargo_picked_up": 23, "overflow_events": 55, "fuel_consumed": 237.89}
{"episode": 1, "steps": 25, "total_reward": 3.17, "cargo_delivered": 7, "cargo_picked_up": 20, "overflow_events": 95, "fuel_consumed": 187.473}

PPO baseline trained for 20000 timesteps with 2 environments:

{"episodes": 10, "avg_reward": -695.246, "avg_deliveries": 71.6, "avg_overflows": 4292.3}

Interpretation:

• —the environment is trainable end to end
• —the PPO agent already learns to increase completed deliveries
• —overflow pressure remains high, which makes the environment non-trivial and leaves room for better policies

How to present this in the hackathon

Pitch it as a logistics control benchmark, not just a training script.

Use this framing:

1. 1.Manufacturers lose throughput when finished goods pile up at loading docks.
2. 2.This environment turns that pain point into an RL problem with explicit observations, actions, and shaped rewards.
3. 3.The contribution is a reusable, trainable benchmark for outbound pickup logistics with a working PPO baseline.

Submission checklist

• —environment code in razorpay_itch_env/
• —OpenEnv server entrypoint in openenv_app.py
• —root inference script in inference.py
• —root deployment manifest in openenv.yaml
• —root container file in Dockerfile
• —training script in train.py
• —evaluation script in evaluate.py
• —screenshots in media/
• —trained checkpoint in artifacts/
• —solution summary in SOLUTION.md

Next improvements

• —add multi-stop routing instead of single-pickup dispatch
• —model travel time directly instead of instant movement
• —add hard delivery deadlines and SLA penalties
• —add curriculum generation for easy-to-hard scenarios
• —benchmark PPO against hand-built dispatch heuristics