shashanks/medical_coding

1---2title: Medical Coding Auditor3emoji: 🏥4colorFrom: blue5colorTo: indigo6sdk: docker7app_port: 78608pinned: false9tags:10  - openenv11base_path: /web12---13 14# Medical Coding Auditor — OpenEnv Environment15 16A real-world reinforcement learning environment that simulates a **hospital pre-bill compliance review**. The AI agent acts as a **Medical Coding Auditor**, reviewing proposed ICD-10-CM and CPT billing codes against clinical notes and patient demographics to identify sequencing violations, demographic mismatches, and mutually exclusive code conflicts.17 18## Motivation19 20Medical coding errors cost the US healthcare system billions of dollars annually in claim denials, penalties, and compliance audits. A skilled auditor must cross-reference ICD-10-CM guidelines, CMS NCCI (National Correct Coding Initiative) edit tables, and patient demographics — a task ideal for evaluating multi-step reasoning in language model agents.21 22This environment tests:23- **Demographic awareness** — recognizing codes inapplicable to a patient's sex or age24- **Guideline reasoning** — interpreting official ICD-10-CM Excludes1/2 notes25- **Regulatory knowledge** — applying CMS NCCI PTP (Procedure-to-Procedure) bundling rules26- **7th character specificity** — validating encounter-type extensions for injury codes27- **Evidence grounding** — extracting exact clinical note text to support coding decisions28- **Disambiguation** — asking physician clarification questions when documentation is ambiguous29- **Hallucination resistance** — penalizing references to codes outside the proposed set30 31---32 33## Action Space34 35The agent selects one action per step from the following tools:36 37| `action_type` | Required Fields | Description |38|---|---|---|39| `query_guideline` | `code` | Look up official ICD-10-CM or CPT coding guidelines, Excludes1/2 notes, and gender restrictions for a specific code. |40| `check_ncci_edits` | `code1`, `code2` | Check the CMS NCCI PTP edit table to determine if two CPT codes have a bundling conflict (mutually exclusive). |41| `flag_error` | `code`, `error_type`, `justification` | Record a confirmed coding error in the draft audit report. |42| `ask_clarifying_question` | `question` | Ask the simulated physician for missing clinical information when the note is ambiguous. Returns a deterministic physician response. |43| `extract_evidence` | `evidence_text` | Highlight an exact text span from the clinical note as documentary evidence for a coding decision. Must be a verbatim substring of the note. |44| `submit_audit` | *(none)* | End the episode and submit the draft report for deterministic grading. |45 46### `error_type` values for `flag_error`:47 48| Value | Description |49|---|---|50| `demographic_mismatch` | Code is inapplicable to the patient's sex or age (e.g., maternity code for a male patient). |51| `excludes1_conflict` | Two mutually exclusive ICD-10-CM diagnosis codes are billed together (Excludes1 rule violation). |52| `ncci_edit` | Two CPT codes have a CMS NCCI PTP bundling conflict; the component service is billed separately from its comprehensive code. |53| `specificity_error` | Wrong 7th character extension (e.g., "initial encounter" used for a follow-up visit). |54| `untraceable_code` | The code does not exist in any official ICD-10-CM or CPT code set. |55 56---57 58## Observation Space59 60Each step returns a `MedicalCodingObservation` with the following fields:61 62| Field | Type | Description |63|---|---|---|64| `task_id` | `str` | Current task identifier. |65| `difficulty` | `str` | `easy`, `medium`, `hard`, or `expert`. |66| `patient_demographics` | `dict` | Age, sex, MRN, insurance carrier. |67| `clinical_note` | `str` | Unstructured clinical note documenting the encounter. |68| `proposed_codes` | `dict[code → {description, code_type}]` | The billing codes to audit. |69| `draft_report` | `list[{code, error_type, justification, step}]` | Errors flagged so far. |70| `tool_result` | `str` | Text output of the last action (guideline text, NCCI result, physician response, etc.). |71| `step_count` | `int` | Steps taken this episode. |72| `codes_queried` | `list[str]` | Codes already queried via `query_guideline` (loop detection). |73| `pairs_checked` | `list[str]` | NCCI pairs already checked (formatted as `code1\|code2`). |74| `clarifications_asked` | `list[str]` | Clarifying questions asked via `ask_clarifying_question` this episode. |75| `extracted_evidence` | `list[str]` | Text spans extracted from the clinical note via `extract_evidence` this episode. |76| `last_action_error` | `str \| null` | Error message if the last action was invalid. |77| `grader_score` | `float \| null` | Final grader score `[0.0, 1.0]` (set only after `submit_audit`). |78| `episode_metrics` | `dict \| null` | Detailed trajectory stats (populated on `done=True`): trajectory length, tool failure rate, investigation coverage, flag precision/recall, avg reward per step, investigation-before-flag rate, clarification count. |79| `reward` | `float` | Reward for the last action. |80| `done` | `bool` | Whether the episode has ended. |81 82---83 84## Tasks85 86Five tasks spanning four difficulty levels:87 88### Task 1 — Easy: Demographic Mismatch (`easy_demographic`)89 90**Scenario:** A 34-year-old **male** patient's annual physical exam chart contains a maternity-specific ICD-10 code (`O80 — Encounter for full-term uncomplicated delivery`) alongside valid codes `E11.9` and `Z00.00`.91 92**Objective:** Query the guideline for `O80`, recognize the female-only restriction, and flag the demographic mismatch before submitting the audit.93 94**Expected error:** `O80` → `demographic_mismatch`95 96**Max steps:** 1097 98---99 100### Task 2 — Medium: NCCI PTP Bundling Conflict (`medium_ncci_conflict`)101 102**Scenario:** A 67-year-old female's cardiology billing includes `93306` (complete transthoracic echocardiography) and `93307` (limited TTE), which are subject to a CMS NCCI Procedure-to-Procedure edit — a classic unbundling violation. Also includes a valid `99213` office visit.103 104**Objective:** Use `check_ncci_edits` to identify the bundling conflict, then flag the violation.105 106**Expected error:** `93306` → `ncci_edit`107 108**Max steps:** 15109 110---111 112### Task 3 — Medium: Excludes1 Conflict (`medium_excludes1`)113 114**Scenario:** A 58-year-old female presents with an acute COPD exacerbation. The proposed codes list both `J44.1` (COPD with exacerbation) and `J45.20` (mild intermittent asthma), which have an Excludes1 (mutually exclusive) relationship. A valid `Z87.891` history code is present as a false-positive trap.115 116**Objective:** Query guidelines to identify the Excludes1 restriction, read the clinical note to confirm the asthma diagnosis was superseded, and flag the conflict.117 118**Expected error:** `J44.1` → `excludes1_conflict`119 120**Max steps:** 15121 122---123 124### Task 4 — Hard: 7th Character Specificity + Untraceable Code (`hard_specificity_untraceable`)125 126**Scenario:** A 28-year-old male presents for a **follow-up visit** (6 weeks post-fracture). The proposed codes include:127- `S52.501A` — uses 7th character `A` (initial encounter) for what the clinical note explicitly documents as a subsequent/follow-up encounter (should be `S52.501D`).128- `Z99.999` — a code that does not exist in any official ICD-10-CM code set.129 130**Objective:** Read the clinical note carefully, verify `Z99.999` is untraceable, and flag both errors.131 132**Expected errors:** `S52.501A` → `specificity_error` AND `Z99.999` → `untraceable_code`133 134**Max steps:** 20135 136---137 138### Task 5 — Expert: Multi-Error Complex Encounter (`expert_multi_error`)139 140**Scenario:** A 65-year-old male has a comprehensive cardiology + endocrinology visit. The proposed 6 codes contain two distinct errors — an Excludes1 conflict between Type 1 and Type 2 diabetes (`E10.9` + `E11.9`) and an NCCI PTP bundling violation between echocardiography codes (`93306` + `93308`). `99214` and `M79.621` are valid and must NOT be flagged.141 142**Objective:** Identify both errors without false-positiving the valid codes.143 144**Expected errors:** `E10.9` → `excludes1_conflict` AND `93306` → `ncci_edit`145 146**Max steps:** 25147 148---149 150## Reward Function151 152The environment provides **dense, multi-signal rewards** throughout each episode:153 154| Action | Condition | Reward |155|---|---|---|156| `query_guideline` | Valid code, first query | `+0.10` |157| `query_guideline` | Code NOT in proposed set (hallucination) | `−0.50` |158| `query_guideline` | Code already queried (loop) | `−0.10` |159| `check_ncci_edits` | Valid pair, first check — no edit found | `+0.05` |160| `check_ncci_edits` | Valid pair, first check — edit found | `+0.10` |161| `check_ncci_edits` | Any code NOT in proposed set | `−0.50` |162| `ask_clarifying_question` | Relevant question (matches physician pool) | `+0.15` |163| `ask_clarifying_question` | Irrelevant or repeated question | `−0.10` |164| `extract_evidence` | Span is in note AND matches expected evidence | `+0.05` |165| `extract_evidence` | Span is in note but irrelevant | `±0.00` |166| `extract_evidence` | Span NOT in clinical note (hallucination) | `−0.05` |167| `flag_error` | Correct code + correct `error_type` (conflict type) | `+0.30 × rarity` |168| `flag_error` | Correct code + correct `error_type` (other) | `+0.20 × rarity` |169| `flag_error` | Correct code + prior evidence extracted | additional `+0.05` |170| `flag_error` | Correct code + justification contains key terms | additional `+0.05` |171| `flag_error` | Correct code + investigated before flagging | additional `+0.05` |172| `flag_error` | Correct code + no prior investigation | `−0.05` |173| `flag_error` | Correct code + wrong error_type (close miss) | `−0.05` |174| `flag_error` | Correct code + wrong error_type (far miss) | `−0.10` |175| `flag_error` | False positive (code has no expected error) | `−0.20` |176| `flag_error` | Code NOT in proposed set | `−0.50` |177| `submit_audit` | Terminal: grader score − FP penalty + efficiency bonus | `[0.0 – 1.0]` |178 179**Rarity multipliers** (Asymmetric Loss — rare errors rewarded more):180`demographic_mismatch=1.0×`, `ncci_edit/excludes1_conflict=1.2×`, `untraceable_code=1.3×`, `specificity_error=1.5×`181 182**HERON-style hierarchical wrong-type penalty:** close misclassifications (e.g. `excludes1_conflict` ↔ `ncci_edit`) penalized less (`−0.05`) than cross-category misclassifications (`−0.10`).183 184### Grader (terminal)185 186The grader runs deterministically at `submit_audit`:187 188- **Full credit (1.0 × rarity):** correct code + correct `error_type`189- **HERON partial credit (0.4–0.55 × rarity):** correct code + wrong `error_type` (scaled by conceptual distance)190- **No credit (0.0):** code not flagged at all191- **Score = `sum(credits) / sum(rarity weights)`** → range `[0.0, 1.0]`192- **Efficiency bonus:** `+0.1 × (max_steps − steps_used) / max_steps`193- **False positive penalty:** `−0.15 × false_positive_count`194- **Thorough review bonus:** `+0.05` if all proposed codes were investigated before submit195 196---197 198## Setup & Usage199 200### Prerequisites201 202- Python ≥ 3.10203- Docker (for containerized deployment)204- `uv` or `pip` for dependency management205 206### Local Development207 208```bash209cd medical_coding_env210 211# Install dependencies212pip install "openenv-core[core]>=0.2.2" openai213 214# Start the server215uvicorn server.app:app --host 0.0.0.0 --port 7860216# or with uv:217uv run server218 219# Validate the environment220openenv validate .221```222 223### Docker224 225```bash226cd medical_coding_env227 228# Build (using root Dockerfile — recommended)229docker build -t medical-coding-env:latest .230 231# Run232docker run -p 7860:7860 medical-coding-env:latest233```234 235### Running Inference236 237```bash238export HF_TOKEN="your_hf_token"239export API_BASE_URL="https://router.huggingface.co/v1"240export MODEL_NAME="Qwen/Qwen2.5-72B-Instruct"241export ENV_BASE_URL="http://localhost:7860"242 243python inference.py244```245 246### API Usage247 248```python249import httpx250 251# Reset to a specific task252resp = httpx.post("http://localhost:7860/reset", json={"task_id": "easy_demographic"})253obs = resp.json()["observation"]254print(obs["clinical_note"])255 256# Execute an action257resp = httpx.post("http://localhost:7860/step", json={258    "action": {259        "action_type": "query_guideline",260        "code": "O80"261    }262})263print(resp.json()["observation"]["tool_result"])264```265 266---267 268## Baseline Scores269 270Evaluated with `Qwen/Qwen2.5-72B-Instruct` via HuggingFace router at temperature 0.2:271 272| Task | Difficulty | Expected Errors | Score |273|---|---|---|---|274| `easy_demographic` | Easy | 1 | ~0.75 |275| `medium_ncci_conflict` | Medium | 1 | ~0.65 |276| `medium_excludes1` | Medium | 1 | ~0.65 |277| `hard_specificity_untraceable` | Hard | 2 | ~0.55 |278| `expert_multi_error` | Expert | 2 | ~0.40 |279| **Average** | — | — | **~0.60** |280 281*(Scores are reproducible given the deterministic grader and fixed random seed.)*282 283---284 285## Project Structure286 287```288medical_coding_env/289├── openenv.yaml               # OpenEnv spec manifest290├── pyproject.toml             # Package config + uv/pip dependencies291├── uv.lock                    # Reproducible dependency lockfile292├── Dockerfile                 # Root Dockerfile (build context = project root)293├── .dockerignore              # Excludes .git/, __pycache__, docs/ from image294├── models.py                  # Action + Observation Pydantic models295├── client.py                  # EnvClient (typed HTTP/WS client)296├── inference.py               # Baseline inference script297├── README.md                  # This file298├── data/299│   ├── ground_truth_cases.json  # ICD-10/CPT guidelines + 5 task scenarios300│   └── case_generator.py        # Procedural random case generator (seeded)301└── server/302    ├── __init__.py303    ├── environment.py         # MedicalCodingEnvironment (reset/step/state)304    ├── app.py                 # FastAPI app (create_app wrapper)305    ├── requirements.txt       # Server runtime deps (pip fallback)306    └── Dockerfile             # Multi-stage Dockerfile (openenv-base pattern)307```308 309---310 311## License312 313Open source under the MIT License. The ICD-10-CM codes and NCCI edit examples used in this environment are based on publicly available CMS government data.314