shashanks/medical_coding

Medical Coding Auditor — OpenEnv Environment

A real-world reinforcement learning environment that simulates a hospital pre-bill compliance review. The AI agent acts as a Medical Coding Auditor, reviewing proposed ICD-10-CM and CPT billing codes against clinical notes and patient demographics to identify sequencing violations, demographic mismatches, and mutually exclusive code conflicts.

Motivation

Medical coding errors cost the US healthcare system billions of dollars annually in claim denials, penalties, and compliance audits. A skilled auditor must cross-reference ICD-10-CM guidelines, CMS NCCI (National Correct Coding Initiative) edit tables, and patient demographics — a task ideal for evaluating multi-step reasoning in language model agents.

This environment tests:

• —Demographic awareness — recognizing codes inapplicable to a patient's sex or age
• —Guideline reasoning — interpreting official ICD-10-CM Excludes1/2 notes
• —Regulatory knowledge — applying CMS NCCI PTP (Procedure-to-Procedure) bundling rules
• —7th character specificity — validating encounter-type extensions for injury codes
• —Evidence grounding — extracting exact clinical note text to support coding decisions
• —Disambiguation — asking physician clarification questions when documentation is ambiguous
• —Hallucination resistance — penalizing references to codes outside the proposed set

Action Space

The agent selects one action per step from the following tools:

error_type values for flag_error:

Observation Space

Each step returns a MedicalCodingObservation with the following fields:

Tasks

Five tasks spanning four difficulty levels:

Task 1 — Easy: Demographic Mismatch (easy_demographic)

Scenario: A 34-year-old male patient's annual physical exam chart contains a maternity-specific ICD-10 code (O80 — Encounter for full-term uncomplicated delivery) alongside valid codes E11.9 and Z00.00.

Objective: Query the guideline for O80, recognize the female-only restriction, and flag the demographic mismatch before submitting the audit.

Expected error: O80 → demographic_mismatch

Max steps: 10

Task 2 — Medium: NCCI PTP Bundling Conflict (medium_ncci_conflict)

Scenario: A 67-year-old female's cardiology billing includes 93306 (complete transthoracic echocardiography) and 93307 (limited TTE), which are subject to a CMS NCCI Procedure-to-Procedure edit — a classic unbundling violation. Also includes a valid 99213 office visit.

Objective: Use check_ncci_edits to identify the bundling conflict, then flag the violation.

Expected error: 93306 → ncci_edit

Max steps: 15

Task 3 — Medium: Excludes1 Conflict (medium_excludes1)

Scenario: A 58-year-old female presents with an acute COPD exacerbation. The proposed codes list both J44.1 (COPD with exacerbation) and J45.20 (mild intermittent asthma), which have an Excludes1 (mutually exclusive) relationship. A valid Z87.891 history code is present as a false-positive trap.

Objective: Query guidelines to identify the Excludes1 restriction, read the clinical note to confirm the asthma diagnosis was superseded, and flag the conflict.

Expected error: J44.1 → excludes1_conflict

Max steps: 15

Task 4 — Hard: 7th Character Specificity + Untraceable Code (hard_specificity_untraceable)

Scenario: A 28-year-old male presents for a follow-up visit (6 weeks post-fracture). The proposed codes include:

• —S52.501A — uses 7th character A (initial encounter) for what the clinical note explicitly documents as a subsequent/follow-up encounter (should be S52.501D).
• —Z99.999 — a code that does not exist in any official ICD-10-CM code set.

Objective: Read the clinical note carefully, verify Z99.999 is untraceable, and flag both errors.

Expected errors: S52.501A → specificity_error AND Z99.999 → untraceable_code

Max steps: 20

Task 5 — Expert: Multi-Error Complex Encounter (expert_multi_error)

Scenario: A 65-year-old male has a comprehensive cardiology + endocrinology visit. The proposed 6 codes contain two distinct errors — an Excludes1 conflict between Type 1 and Type 2 diabetes (E10.9 + E11.9) and an NCCI PTP bundling violation between echocardiography codes (93306 + 93308). 99214 and M79.621 are valid and must NOT be flagged.

Objective: Identify both errors without false-positiving the valid codes.

Expected errors: E10.9 → excludes1_conflict AND 93306 → ncci_edit

Max steps: 25

Reward Function

The environment provides dense, multi-signal rewards throughout each episode:

Rarity multipliers (Asymmetric Loss — rare errors rewarded more): demographic_mismatch=1.0×, ncci_edit/excludes1_conflict=1.2×, untraceable_code=1.3×, specificity_error=1.5×

HERON-style hierarchical wrong-type penalty: close misclassifications (e.g. excludes1_conflict ↔ ncci_edit) penalized less (−0.05) than cross-category misclassifications (−0.10).

Grader (terminal)

The grader runs deterministically at submit_audit:

• —Full credit (1.0 × rarity): correct code + correct error_type
• —HERON partial credit (0.4–0.55 × rarity): correct code + wrong error_type (scaled by conceptual distance)
• —No credit (0.0): code not flagged at all
• —Score = `sum(credits) / sum(rarity weights)` → range [0.0, 1.0]
• —Efficiency bonus: +0.1 × (max_steps − steps_used) / max_steps
• —False positive penalty: −0.15 × false_positive_count
• —Thorough review bonus: +0.05 if all proposed codes were investigated before submit

Setup & Usage

Prerequisites

• —Python ≥ 3.10
• —Docker (for containerized deployment)
• —uv or pip for dependency management

Local Development

cd medical_coding_env

# Install dependencies
pip install "openenv-core[core]>=0.2.2" openai

# Start the server
uvicorn server.app:app --host 0.0.0.0 --port 7860
# or with uv:
uv run server

# Validate the environment
openenv validate .

Docker

cd medical_coding_env

# Build (using root Dockerfile — recommended)
docker build -t medical-coding-env:latest .

# Run
docker run -p 7860:7860 medical-coding-env:latest

Running Inference

export HF_TOKEN="your_hf_token"
export API_BASE_URL="https://router.huggingface.co/v1"
export MODEL_NAME="Qwen/Qwen2.5-72B-Instruct"
export ENV_BASE_URL="http://localhost:7860"

python inference.py

API Usage

import httpx

# Reset to a specific task
resp = httpx.post("http://localhost:7860/reset", json={"task_id": "easy_demographic"})
obs = resp.json()["observation"]
print(obs["clinical_note"])

# Execute an action
resp = httpx.post("http://localhost:7860/step", json={
    "action": {
        "action_type": "query_guideline",
        "code": "O80"
    }
})
print(resp.json()["observation"]["tool_result"])

Baseline Scores

Evaluated with Qwen/Qwen2.5-72B-Instruct via HuggingFace router at temperature 0.2:

(Scores are reproducible given the deterministic grader and fixed random seed.)

Project Structure

medical_coding_env/
├── openenv.yaml               # OpenEnv spec manifest
├── pyproject.toml             # Package config + uv/pip dependencies
├── uv.lock                    # Reproducible dependency lockfile
├── Dockerfile                 # Root Dockerfile (build context = project root)
├── .dockerignore              # Excludes .git/, __pycache__, docs/ from image
├── models.py                  # Action + Observation Pydantic models
├── client.py                  # EnvClient (typed HTTP/WS client)
├── inference.py               # Baseline inference script
├── README.md                  # This file
├── data/
│   ├── ground_truth_cases.json  # ICD-10/CPT guidelines + 5 task scenarios
│   └── case_generator.py        # Procedural random case generator (seeded)
└── server/
    ├── __init__.py
    ├── environment.py         # MedicalCodingEnvironment (reset/step/state)
    ├── app.py                 # FastAPI app (create_app wrapper)
    ├── requirements.txt       # Server runtime deps (pip fallback)
    └── Dockerfile             # Multi-stage Dockerfile (openenv-base pattern)

License

Open source under the MIT License. The ICD-10-CM codes and NCCI edit examples used in this environment are based on publicly available CMS government data.