VEDAGI1/Medica_DecisionSupportAI

ClarityOps 🍁 — Privacy-First Healthcare Analytics Copilot

   

A two-phase AI decision-support app for healthcare operations that turns scenario text and uploaded files into structured, auditable, privacy-aware insights.

ClarityOps is a Gradio-based analytics assistant built for healthcare decision support, especially privacy-sensitive operational use cases such as wait times, capacity planning, cost modeling, utilization analysis, workforce questions, and service improvement scenarios. It combines LLM-guided reasoning with deterministic calculations, schema validation, sandboxed execution, audit checks, and privacy guardrails to reduce hallucination risk and improve traceability.

The project is configured for local use, Docker deployment, and Hugging Face Spaces, with a workflow centered on clarify first, analyze second.

📌 Executive Reporting Standard

As of the latest hardening pass, every final output is forced through services/executive_output_formatter.py.

• —Board-ready by default: concise executive summary, direct recommendations, and a mandatory Board Recommendation box
• —Exact-value fidelity: benchmark, equity, and disparity reports preserve source numbers without invented scaling or softening
• —Disparity-first prioritization: the largest validated gaps are surfaced first
• —Indigenous signal surfaced explicitly when uploaded source data contains Indigenous response volume or disparity evidence
• —Current repository test surface: tests/ currently contains 161 discovered test methods across routing, privacy, forecasting, sandbox, and reporting modules

✨ Features: Complete Feature Inventory

1. Intelligent Query Understanding & Decomposition

• —Language Detection: Auto-detects user's language (English, French, Spanish, etc.) and sets prompt language accordingly
• —Intent-Aware Query Profiling: Classifies requests into route families such as simple_text_reasoning, structured_exec_report, strategic_multidomain, and predictive paths before routing
• —Scenario Complexity Detection: Distinguishes single-question prompts from multi-part scenarios with "Evaluation Questions"
• —Smart Decomposition: Multi-part scenarios are split into focused sub-questions, each routed independently to the appropriate template
• —Context Preservation: Numeric facts and domain context from the original scenario are preserved and injected into each sub-analysis
• —Sheet-Aware Data Catalog: Workbook sheets, sections, and candidate shared keys are appended to schema context so multi-file analysis preserves joins and workbook structure
• —Intent-Aware Clarifications: Clean board-report / executive-summary workbook prompts are not over-clarified, while truly strategic or predictive cases still ask targeted Phase 1 questions
• —Augmented Instructions: Sub-analysis instructions are enriched with relevant facts (e.g., "3 teams", "$1.2M budget") to maintain consistency

2. Deterministic Analytics Templates (7 Categories)

• —Template Matching: 7 deterministic frameworks encode common healthcare questions (benchmarking, operations, cost, workforce, utilization, trends, forecasting)
• —Data-Shape Driven: Templates activate based on schema fingerprinting + keyword matching—not hardcoded to specific scenarios
• —Specialized Builders: Each template has a Python builder that generates deterministic code (no LLM in calculation layer)
• —Avoids hallucination in math (e.g., "calculate capacity = teams × hours × utilization")
• —Enables audit traceability (code matches published formula)
• —High Confidence: Deterministic template outputs score 0.90+, LLM fallback only if no match

3. Automated File Parsing & Schema Resolution

• —Multi-Format Support: CSV, XLSX/XLS, PDF, and DOCX are automatically detected and parsed
• —Structural Parser: Extracts tables, headers, sections, and metadata from each file
• —Column Fingerprinting: Detects column semantic roles (financial, clinical, operational) based on names and content
• —Embedded Data Detection: Identifies data embedded in narrative text, PDFs, or images (OCR-ready pattern)
• —Fuzzy Column Matching: Generated code handles aliases or typos (e.g., "Total$Cost" matched to "total_cost")

4. Sandboxed Code Execution with Safety Controls

• —Restricted Imports: Only pandas, numpy, scipy, sklearn, json, math, datetime, re allowed; blocks os, subprocess, socket, pickle
• —Output Capture: All STDOUT/STDERR captured; sandbox wall-clock timeout enforcement prevents runaway loops
• —Auto-Repair: Catches 10+ common pandas/numpy errors and retries with fixes:
• —df["col"] → df.loc[:, "col"] (correct indexing)
• —Missing .reset_index() on groupby results
• —Type mismatches in filters
• —No Side Effects: No file I/O, no external API calls within sandbox; 100% reproducible

5. Multi-Layer Validation Pipeline

• —JSON Schema Validation: All outputs validated against strict Pydantic models and JSON schemas; enforces required fields, types, value ranges
• —Math Verification: Checks that formulas are correct (e.g., capacity = teamsize × teamcount × time_window)
• —Unit Consistency: Detects and converts mixed units (days ↔ hours, $/client ↔ $/1000clients, occupancy as % or decimal)
• —Policy Guardrails: Medical best-practices enforced (staffing ratios, realistic cost ranges, clinical outcome bounds)
• —Data Integrity: Verifies all referenced DataFrames exist, numeric ranges are plausible, outputs match input data
• —Truncation Detection: Identifies incomplete LLM outputs and auto-continues them seamlessly

6. Canadian Privacy Compliance (PHIPA/PIPEDA)

• —Dual-Phase Compliance Scanning:
• —Regex phase: SSN, MRN, phone, email, postal code, DOB patterns
• —NER phase (optional Presidio): Named entities (names, facility names, conditions) in narrative
• —Intelligent Aggregate Detection: Allows population-level reporting ("42% of cohort") while blocking individual-level PHI ("John Smith had A1C 9.2")
• —Small-Cell Suppression: Cells with counts < 5 replaced with [SUPPRESSED] (configurable threshold)
• —PHI Redaction: Identified elements replaced with [REDACTED_SSN], [REDACTED_DOB], etc.
• —Date Shifting: Session-consistent date offset (30–366 days) applied to all dates for de-identification
• —Pre-LLM Redaction (Optional): Can redact before sending to Cohere to keep data local (optional extra privacy layer)

7. Independent Audit Agent with Quality Assurance

• —Confidence Scoring (0.0–1.0):
• —Template Output Score: 0.90 floor (deterministic, no LLM in code generation)
• —LLM Output Score: 0.50–0.80 based on code structure, completeness, data usage
• —Self-Healing Retry Loop:
• —Automatic retry if confidence < 0.95 (max 2 total attempts)
• —Best-Response Preservation: Tracks highest-scoring attempt; never downgrades to lower-confidence rewrites
• —Early Exit: Deterministic template outputs preserved without retry if already high-confidence
• —Variance Detection: Flags unstable results if audit score variance exceeds 0.15 over last 10 analyses
• —Per-Call Latency Tracking: All phases timed (parsing, routing, sandbox, audit, compliance, reporting)
• —Formatted Audit Badge:

  ✅ VERIFIED | Confidence: 0.95 | Method: scenario_template | Latency: 1,240ms
  🔒 PASSED | PHIPA/PIPEDA: compliant | Small-cell suppression: applied

8. Role-Based Access Control (RBAC) & Session Management

• —Four Roles:
• —Admin: Full access, user management, audit log export, policy updates
• —Analyst: Run analyses, upload data, view results, no user management
• —Viewer: View results only, no data upload or analysis
• —Service: Automated systems (benchmarks, CI/CD), run + view only
• —Session Management:
• —Configurable timeout (default 8 hours)
• —Session history persistence (PERSIST_HISTORY flag)
• —Secure session cleanup on logout
• —Per-user audit trails
• —User Store: JSON-based (upgradeable to database); supports password hashing with salt

9. LLM Routing with Fault Tolerance

• —Circuit Breaker: Prevents hammering a failed API; enters "open" state after 5 failures, "half-open" for recovery probes
• —Fail-fast behavior (no hanging) during outages
• —Gradual recovery testing when service restarts
• —Request Deduplication: Prevents duplicate calls on double-click (5-minute in-memory cache)
• —Rate Limit Awareness: Reads Cohere response headers; pre-emptive backoff if near limit
• —Exponential Retry with Jitter: Transient errors retried with increasing delays (1s, 2s, 4s, ...)
• —Model Fallback Chain:
• —command-a-03-2025 (primary)
• —command-a-03 (fallback)
• —Optional local LLM if USE_OPEN_FALLBACKS enabled
• —Metric Tracking: Per-call latency, token usage (prompt + completion), estimated cost

10. Advanced Report Generation

• —Structured Markdown Output: Headers, tables, formulas, narrative summary
• —ExecutiveOutputFormatter Enforcement: Final responses are rewritten into concise, decisive, board-grade outputs before delivery
• —Mandatory Board Recommendation Box: Each executive-style report ends with a direct action statement for leadership
• —Exact-Value Preservation: Benchmark, equity, and patient-experience reports keep validated numeric values intact rather than inventing rounding or scaling
• —Disparity-First Reporting: The largest validated gaps are promoted to the top of the report
• —Indigenous Signal Surfacing: Indigenous response volumes and signals are called out explicitly when present in the source workbook
• —Truncation Handling: Auto-detects incomplete LLM outputs and auto-continues asynchronously
• —Report Verification: Cross-checks numbers in report against source data; flags inconsistencies
• —Deterministic Summaries (for common patterns):
• —Community Prioritization: Rankings with rationale
• —Operational: Bottleneck identification, capacity recommendations
• —Financial: Cost breakdowns, benchmark comparisons
• —Clinical: Outcome deltas, benefit projections
• —Language-Aware: Output language matches input language (if translation template available)
• —Audit Trail: Every report is linked to analysis ID, user, timestamp, prompt hash, filenames, and schema-level metadata

11. Privacy Protections & Session Safety

• —PHI_MODE Flag: Turns privacy protections on/off globally
• —Session Cleanup: Temporary files, analysis caches, history cleared on logout or timeout
• —Audit Trail Encryption: Persistent audit records are encrypted at rest using key material from CLARITYOPS_AUDIT_KEY
• —Secure Password Hashing: SHA-256 with salts; passwords never stored in plaintext
• —SIEM / Compliance Export: Encrypted audit records can be decrypted and exported as JSON for downstream review tooling

12. Policy & Guidance Integration

• —Policy Index Building: build_policy_index.py creates retrieval index from policies/ folder (medical guardrails, best practices)
• —RAG Support: Policies can be injected into prompts to guide analysis (optional, configurable)
• —Medical Guardrails: Policies in policies/medical_guardrails.md encode clinical constraints (e.g., realistic clinical outcomes, staffing ratios)
• —Canadian Regulatory Guidance: PHIPA/PIPEDA reference docs; audit log compliance

13. Extensible Validation & Enrichment

• —Analytical Primitives: Reusable calculation blocks (moving averages, percentile calculations, outlier detection)
• —Analytical Enrichment: Post-processing layer adds derived metrics (e.g., "% change from baseline", "days to breakeven")
• —Deterministic Post-Processor: Applies classification logic (e.g., "capacity_status" → "constrained" vs "adequate" vs "excess")
• —Theme Engine: Accumulates key findings from multi-part scenarios into a cohesive "storyline"

🩺 What Problem It Solves

Healthcare teams often need fast answers to operational questions, but raw spreadsheets, narrative briefs, and privacy constraints make analysis slow and error-prone. ClarityOps helps by:

• —turning messy inputs into structured analysis,
• —asking for missing context before making assumptions,
• —calculating key metrics in code instead of guessing them in prose,
• —checking outputs for privacy and accuracy issues, and
• —returning a report that is easier to review and trust.

Important: ClarityOps is a decision-support tool. It is not a substitute for clinical judgment, legal review, or organizational privacy/governance approval.

🧠 System Architecture

Complete Analysis Workflow

User Scenario + Files
    ↓
┌─────────────────────────────────────────────────────────────────┐
│ PHASE 1: INPUT & UNDERSTANDING                                  │
├─────────────────────────────────────────────────────────────────┤
│ • File Parsing (CSV, XLSX, PDF, DOCX)                           │
│ • Schema Fingerprinting (detect domain: clinical, financial…)   │
│ • Sheet-Aware Data Catalog + Join Hints                         │
│ • Language Detection (automatic, sets prompt language)          │
│ • Query Profiling (exec report vs. strategic vs. predictive)    │
│ • Complexity Detection (single vs. multi-part scenarios)        │
│ • Decomposition (splits multi-part into focused sub-questions)  │
│ • PHI Redaction (optional pre-LLM, preserves data semantics)    │
└─────────────────────────────────────────────────────────────────┘
    ↓
┌─────────────────────────────────────────────────────────────────┐
│ PHASE 2: INTELLIGENT ROUTING                                    │
├─────────────────────────────────────────────────────────────────┤
│ • Scenario Template Matching (7 deterministic categories)       │
│   - P0: Group-vs-Benchmark Performance (equity, disparities)    │
│   - P1: Operational Performance (wait times, throughput)        │
│   - P2: Financial/Cost Analysis (ROI, cost per case)            │
│   - P3: Workforce Analytics (staffing, productivity)            │
│   - P4: Utilization/Volume Analysis (admissions, visits)        │
│   - P5: Trend/Time Series (QoQ, YoY trends)                     │
│   - P6: Generic Demand Forecast (time-series forecasting)        │
│ • Keyword Scoring + Schema Fingerprinting (no hardcoding)       │
│ • Specialized Template Builders (deterministic > LLM fallback)  │
└─────────────────────────────────────────────────────────────────┘
    ↓
┌─────────────────────────────────────────────────────────────────┐
│ PHASE 3: CODE GENERATION & EXECUTION                            │
├─────────────────────────────────────────────────────────────────┤
│ • Template Match → Python script output (~95% confidence)       │
│ • No Match → LLM code generation (with fuzzy column matching)   │
│ • Sandbox Execution (restricted imports, no file I/O)           │
│ • Fuzzy Column Resolution (handles typos, aliases)              │
│ • Auto-Repair (pandas common errors, JSON syntax)               │
└─────────────────────────────────────────────────────────────────┘
    ↓
┌─────────────────────────────────────────────────────────────────┐
│ PHASE 4: VALIDATION PIPELINE                                    │
├─────────────────────────────────────────────────────────────────┤
│ • JSON Schema Validation (against schemas/ *.json)              │
│ • Math Verification (capacity = formula, cost = price × qty)    │
│ • Unit Consistency (all $ amounts consistent, days → hours)     │
│ • Policy Validation (medical guardrails, best-practice checks)  │
│ • Data Verification (numbers match source files)                │
│ • Truncation Detection (report completed?)                      │
└─────────────────────────────────────────────────────────────────┘
    ↓
┌─────────────────────────────────────────────────────────────────┐
│ PHASE 5: PRIVACY & COMPLIANCE CHECKS                            │
├─────────────────────────────────────────────────────────────────┤
│ • Compliance Agent (PHIPA/PIPEDA: identifies PII, PHI)          │
│ • Presidio NER (optional: catches names, dates, identifiers)    │
│ • Small-Cell Suppression (cells < 5 → [REDACTED])               │
│ • PHI Redaction (SSN, MRN, DOB, phone, email, postal)           │
│ • Date Shifting (session-consistent date offset for privacy)    │
│ • Aggregate Detection (allows population-level reporting)       │
└─────────────────────────────────────────────────────────────────┘
    ↓
┌─────────────────────────────────────────────────────────────────┐
│ PHASE 6: INDEPENDENT AUDIT & QUALITY CHECKS                     │
├─────────────────────────────────────────────────────────────────┤
│ • Confidence Scoring (0.0–1.0, template floor 0.90)             │
│ • Self-Healing Loop (retry if low confidence, max 2 attempts)   │
│ • Best-Response Tracking (preserves highest rank, never downgrades) │
│ • Variance Detection (flags unstable results)                   │
│ • Deterministic Preference (template outputs preserved early)   │
│ • Per-Call Latency Tracking (all phase timings logged)          │
└─────────────────────────────────────────────────────────────────┘
    ↓
┌─────────────────────────────────────────────────────────────────┐
│ PHASE 7: FINAL REPORTING & DELIVERY                             │
├─────────────────────────────────────────────────────────────────┤
│ • Report Generation (markdown with headers, tables, metrics)    │
│ • Continuation Handling (if report was truncated, auto-complete) │
│ • Audit Badge Formatting (confidence level, pass/fail status)   │
│ • Compliance Badge Formatting (PHIPA/PIPEDA pass/fail)          │
│ • Session History Management (persisted or time-limited)        │
│ • Persistent Audit Events (encrypted records written via safe_log/audit_log.py) │
└─────────────────────────────────────────────────────────────────┘

📊 Scenario Template System (Deterministic Layer)

The heart of ClarityOps is the 7-category deterministic template system that uses schema fingerprinting + keyword matching to automatically route queries to pre-built analytical patterns. No scenarios are hardcoded; routing is purely data-driven.

Template Categories

Routing Algorithm

1. 1.Keyword Scoring: Query is matched against keyword sets for each template. Match confidence = (matched keywords) / (total template keywords).
2. 2.Schema Fingerprinting: Each uploaded file is fingerprinted (numeric cols, categorical cols, domain indicators like "a1c", "cost", "wait_time").
3. 3.Column Role Detection: Column names are mapped to semantic roles (e.g., "totalscreeningcost" → financial, "daystofollow_up" → operational).
4. 4.Specialized Builders: For each strong template match (confidence ≥ 0.70), a specialized Python builder is called:
5. 5._build_community_prioritization() → generates ranking logic
6. 6._build_operational() → generates capacity calculations
7. 7._build_financial() → generates cost formulas
8. 8.(etc. for P3, P4, P5, P6)
9. 9.Code Output: Successful builder returns a ScenarioMatch with Python code, confidence score, matched columns.
10. 10.LLM Fallback: If no template matches strongly (confidence < 0.70), the system falls back to LLM-generated code.

🔒 Validation & Safety Layer

ClarityOps implements a four-layer validation stack to prevent hallucination and improve auditability:

Layer 1: Sandbox Execution with Module Restrictions

• —Allowed: json, math, pandas, numpy, scipy, sklearn, datetime, re
• —Blocked: os, sys, subprocess, socket, http, pickle, importlib, eval, exec
• —Column Resolution: Fuzzy matching for misspelled/aliased columns (e.g., "Total$Cost" → "total_cost")
• —Auto-Repair: Catches common pandas errors (df["col"] vs df.loc[:, "col"]) and retries with correction

Layer 2: JSON Schema Validation

All outputs validated against strict schemas in schemas/:

• —analysis_output.schema.json — financial, operational, clinical outputs
• —phase2_output.schema.json — multi-part scenario outputs
• —Validation enforces required fields, correct types, value ranges, preventing silent schema drift

Layer 3: Math & Unit Verification

• —Math Validator: Ensures total = unit_cost × quantity, capacity = team_size × team_count × time_window
• —Unit Validator: Detects and converts mixed units (days ↔ hours, $/client ↔ $/1000clients)
• —Policy Validator: Medical guardrails (e.g., realistic staffing ratios, acceptable cost ranges)

Layer 4: Data Integrity Checks

• —Column Completeness: Verifies all DataFrames referenced in code are actually present
• —Numeric Range Checking: Flags outliers (e.g., wait times > 365 days, occupancy > 150%)
• —Truncation Detection: Reports incomplete (cut-off) LLM outputs and auto-continues them

🛡️ Privacy & Compliance Framework

Canadian PHIPA/PIPEDA Compliance

ComplianceAgent implements two-phase scanning:

1. 1.Regex Phase: Patterns match—SSN (\d{3}-\d{2}-\d{4}), MRN (\d{9}), phone, email, postal codes, DOB
2. 2.NER Phase (optional, Presidio): Named-entity recognition catches clinic names, first names, conditions hidden in narratives
3. 3.Aggregate Data Detection: Distinguishes between individual-level PHI (blocked) and population-level insights (allowed)
4. 4.Individual: "Patient John Smith had A1C 9.2%" → blocked
5. 5.Aggregate: "42% of cohort had A1C > 9" → allowed

PHI Redaction & De-identification

• —Redaction: Replaces identified elements with [REDACTED_SSN], [REDACTED_DOB], etc.
• —Date Shifting: Session-consistent date offset (30–366 days) applied to all dates in outputs
• —Small-Cell Suppression: Cells with counts < 5 are replaced with [SUPPRESSED] to prevent individual re-identification
• —Min Cell Size: Configurable threshold (default 5) for which aggregate cells are suppressed

PHI Output Control

• —PHI_MODE environment variable: Turns redaction on/off
• —REDACT_BEFORE_LLM: Optionally redacts identifiers before sending to external LLM (preserves data utility)
• —ALLOW_EXTERNAL_PHI: Restricts which LLM providers can see sensitive data
• —Session Cleanup: Temporary files and session history cleared on logout or timeout

🏥 Audit & Traceability Layer

Independent Audit Agent with Production Patterns

AuditAgent runs independently on every analysis:

1. 1.Confidence Scoring (0.0–1.0):
2. 2.Deterministic template outputs: floor = 0.90 (no LLM in code path)
3. 3.LLM-generated code: scored based on code readability, data usage, output structure

1. 1.Self-Healing Retry Loop:
2. 2.If initial confidence < 0.95, attempt one retry (max 2 total attempts)
3. 3.Best-Response Tracking: Retains highest-confidence result; never downgrades to lower-scoring rewrites
4. 4.Deterministic Preference: Template outputs preserved without self-healing if already high-confidence

1. 1.Variance Detection:
2. 2.Tracks last 10 audit scores
3. 3.Flags if std dev > 0.15 (indicates unstable/unreliable analysis)

1. 1.Per-Call Latency Tracking:
2. 2.Audit and self-heal calls record per-attempt latency
3. 3.Aggregate audit timing is preserved in the in-memory audit_record

1. 1.Structured Output:
2. 2.Pydantic models enforce output schema (Presidio-optional, graceful fallback)
3. 3.AuditAgent.run() returns (final_response, audit_record) for in-memory consumption by the UI badge formatter
4. 4.AuditAgent does not persist audit records to disk; persistent audit events flow through safe_log into audit_log.py
5. 5.Regression tests enforce that AuditAgent does not touch the filesystem

Audit Badge Format

✅ **Audit: VERIFIED** | Confidence: 0.95 | Method: scenario_template | Latency: 1,240ms
🔒 **Compliance: PASSED** | PHIPA/PIPEDA: compliant | Small-cell suppression: applied

🎬 LLM Routing with Circuit Breaker

llm_router.py implements production-grade LLM coordination:

Circuit Breaker Pattern

• —CLOSED (Normal): Requests flow to Cohere; failures counted in sliding 120s window
• —OPEN (Outage): After 5 failures, all requests rejected immediately (fail-fast, no hanging)
• —HALF_OPEN (Probing): After 60s recovery window, one request allowed to test if service recovered
• —Metric Tracking: Per-call latency, token usage, cost (Cohere prompt + completion tokens)
• —Request Dedup: Prevents duplicate API calls on double-click (in-memory cache, 5min window)

Model Fallback Chain

1. 1.Primary: command-a-03-2025 (fastest, 0.1s latency)
2. 2.Fallback: command-a-03 (slightly cheaper)
3. 3.Can Override: Set COHERE_MODEL_PRIMARY to use different model
4. 4.Local LLM (optional): If USE_OPEN_FALLBACKS enabled and local_llm module available, can fall back to local model

Rate Limit Awareness

• —Reads Cohere response headers for remaining tokens, rate limit window
• —Pre-emptive back-off if rate limit is near
• —Exponential retry with jitter on transient errors

🛠️ Tech Stack

🚀 Quick Start

1) Clone the repository

git clone <your-repo-url>
cd Medica_DecisionSupportAI

2) Create and activate a virtual environment

Windows (PowerShell)

python -m venv .venv
.\.venv\Scripts\Activate.ps1

macOS / Linux

python -m venv .venv
source .venv/bin/activate

3) Install dependencies

pip install -r requirements.txt

4) Configure environment variables

Copy the template and fill in your values:

cp env_template.env .env

On Windows PowerShell:

Copy-Item env_template.env .env

At minimum, set:

• —COHERE_API_KEY
• —CLARITYOPS_AUDIT_KEY
• —CLARITYOPS_ADMIN_PASSWORD (or ADMIN_PASSWORD)

5) Optional setup tasks

Build the policy index if you update files under policies/:

python build_policy_index.py

Create additional application users:

python setup_users.py

6) Run the app

python app.py

Then open:

http://localhost:7860

To generate a temporary public test link (Gradio tunnel), run with share enabled:

Windows (PowerShell)

$env:GRADIO_SHARE = "true"
python app.py

macOS / Linux

GRADIO_SHARE=true python app.py

When enabled, Gradio prints a https://*.gradio.live URL in the terminal.

💻 Usage

Web app workflow

1. 1.Start the app with python app.py
2. 2.Sign in with your configured credentials
3. 3.Upload one or more supported files
4. 4.Ask a healthcare operations or planning question
5. 5.Answer the clarification questions shown in chat
6. 6.Review the final structured report, audit badge, and compliance notes

Example prompts

• —Which departments should we prioritize to reduce median consult wait time?
• —How much capacity do we need to bring occupancy below 90% next month?
• —What is the per-client startup and ongoing cost for this program?
• —Compare utilization across sites and flag the biggest outliers.

CLI / pipeline usage

You can also run the two-phase flow from the command line:

python -m pipeline.main --scenario "Our clinic is seeing rising wait times and limited staff capacity." --name oncology_pilot

Then, after filling in clarifications.json, run Phase 2:

python -m pipeline.main --pack packs/oncology_pilot --phase2

🎮 Demo / Hugging Face Spaces

This repository is already configured for Hugging Face Spaces with:

• —sdk: gradio
• —app_file: app.py
• —sdk_version: 5.44.1

No public live Space URL is included in this repository. If you deploy a public Space, add the link here.

You can create a deployment at: huggingface.co/new-space

For quick external testing without deploying, run locally with GRADIO_SHARE=true to generate a temporary public link.

📁 Detailed Project Structure & Module Inventory

.
├── Core Application Layer
│   ├── app.py                         # Main Gradio UI; session wiring, file uploads, chat history
│   ├── settings.py                    # Runtime config loader (env vars, model settings, defaults)
│   └── requirements.txt               # Python dependencies (pandas, cohere, gradio, etc.)
│
├── Analysis Pipeline
│   ├── services/
│   │   ├── analysis_service.py        # Top-level orchestration for Phase 1 + Phase 2
│   │   ├── analysis_execution_service.py # Routing, decomposition, sandbox execution, retries
│   │   ├── input_preparation.py       # File parsing, schema context, sheet-aware data catalog
│   │   ├── phase1_service.py          # Clarification generation with intent-aware query profiling
│   │   ├── reporting_service.py       # Report assembly, audit/compliance flow, enrichment
│   │   ├── executive_output_formatter.py # Final board-ready formatting, exact-value preservation, Board Recommendation box
│   │   ├── routing_schema.py          # QueryProfile + route selection + retry/output completeness checks
│   │   ├── validation_pipeline.py     # Shared validation path for UI and CLI
│   │   ├── ui_helpers.py              # UI state management (file status, session refresh, Cohere ping)
│   │   └── __init__.py
│   │
│   ├── scenario_templates.py          # 7-category template library (P0–P6 deterministic frameworks)
│   ├── scenario_manager.py            # Multi-question decomposition & sub-analysis sequential execution
│   ├── deterministic_templates.py     # Atomic template builders (financial, operational, clinical, etc.)
│   ├── context_engine.py              # Query understanding & semantic context extraction
│   └── code_generation.py             # LLM code generation (when no template match)
│
├── Routing & LLM Coordination
│   ├── llm_router.py                  # Circuit breaker, rate limiting, model fallback chain
│   └── local_llm.py                   # Optional local LLM support (open-source fallback)
│
├── Input Parsing & Schema Detection
│   ├── structural_parser.py           # Multi-format file parser (CSV, XLSX/XLS, PDF, DOCX)
│   ├── schema.py                      # Column fingerprinting & semantic role detection
│   ├── schema_validation.py           # Schema consistency & DataFrame validation
│   ├── column_resolver.py             # Column name normalization & fuzzy matching
│   ├── embedded_data_and_column_resolution.py  # Extracts embedded tables/data from narrative
│   └── language_utils.py              # Multi-language support (auto-detect, translations)
│
├── Execution & Sandbox
│   ├── sandbox_executor.py            # Restricted code execution (whitelist imports, no file I/O)
│   └── helpers.py                     # Retry logic, error handling, safe logging utilities
│
├── Validation Pipeline
│   ├── validators/
│   │   ├── math_validator.py          # Formula verification (capacity = formula, cost = price × qty)
│   │   ├── schema_validator.py        # Pydantic + JSON schema validation
│   │   ├── unit_validator.py          # Unit consistency (days ↔ hours, $/client ↔ $/1000s)
│   │   ├── policy_validator.py        # Medical guardrails & best-practice constraints
│   │   └── __init__.py
│   │
│   ├── report_engine.py               # Final report generation, truncation detection, verification
│   ├── report_directives.py           # Report template directives & prompt augmentation
│   ├── json_handler.py                # JSON parsing, validation, formatting for reports
│   ├── deterministic_postprocessor.py # Post-processing: classification, theme extraction, enrichment
│   └── services/executive_output_formatter.py # Final executive rewrite layer for board-ready outputs
│
├── Privacy & Compliance
│   ├── phi_protection.py              # PHI redaction, date shifting, small-cell suppression
│   ├── privacy.py                     # Safety filter, refusal replies for unsafe queries
│   ├── compliance_agent.py            # Independent PHIPA/PIPEDA compliance scanning (Presidio integration)
│   └── (privacy tests live under tests/test_privacy.py and tests/test_phi_protection.py)
│
├── Audit & Security
│   ├── audit_agent.py                 # Independent audit & quality agent; confidence scoring, self-healing loop
│   ├── audit_log.py                   # Audit log encryption & file management
│   ├── audit_trail.py                 # Audit event logging (analysis start/complete, code generation, etc.)
│   ├── rbac.py                        # Role-based access control (4 roles, session timeout, user store)
│   └── session_cleanup.py             # Session history & temporary file cleanup
│
├── Data Processing & Enrichment
│   ├── analytical_primitives.py       # Reusable calculation blocks (moving averages, percentiles, outliers)
│   ├── analytical_enrichment.py       # Post-processing enrichment (derived metrics, delta calculations)
│   ├── graders/
│   │   ├── rule_grader.py             # Rule-based scoring for classification
│   │   └── __init__.py
│   │
│   ├── session_rag.py                 # Session-scoped retrieval-augmented generation
│   └── column_resolver.py             # (advanced column name resolution & aliases)
│
├── Configuration & Policy
│   ├── core/
│   │   ├── ontology.md                # Data dictionary & semantic definitions
│   │   ├── policy_global.json         # Global policy constraints (staffing ratios, cost bounds, etc.)
│   │   └── __init__.py
│   │
│   ├── policies/                      # Medical guardrails, PHIPA/PIPEDA reference docs
│   │   ├── medical_guardrails.md      # Clinical constraints & best practices
│   │   ├── README.md
│   │   └── (custom org policies)
│   │
│   ├── prompts/                       # LLM prompt templates (system, user, phase1, phase2)
│   │   ├── system_master.txt          # Master system prompt
│   │   ├── system_two_phase.txt       # Two-phase system prompt
│   │   ├── answer_clarifications_template.txt
│   │   ├── extractor_system.txt
│   │   ├── phase2_format.txt
│   │   └── (more templates)
│   │
│   ├── schemas/                       # JSON output schemas for validation
│   │   ├── analysis_output.schema.json
│   │   ├── phase2_output.schema.json
│   │   └── (more schemas)
│   │
│   ├── snapshots/                     # Data snapshots for debugging / auditing
│   │   └── current.json               # Latest analysis snapshot
│   │
│   ├── build_policy_index.py          # Builds retrieval index from policies/ (optional)
│   ├── setup_users.py                 # CLI user creation & management
│   └── env_template.env               # Environment variable template
│
├── CLI & Pipeline Tools
│   ├── pipeline/
│   │   ├── main.py                    # Two-phase CLI entry point (Phase 1 clarification, Phase 2 analysis)
│   │   ├── run_two_phase.py           # Orchestrates Phase 1 & Phase 2 in sequence
│   │   ├── pack_builder.py            # Bundles scenario + data + clarifications into a "pack"
│   │   ├── io_utils.py                # File I/O, pack loading/saving
│   │   └── __init__.py
│   │
│   ├── universal_schema_fix.py        # (schema repair utility, for edge cases)
│   └── (session cleanup validation is covered in tests/test_privacy.py)
│
├── Tests & Quality Assurance
│   ├── tests/
│   │   ├── test_routing_and_ingestion.py  # Routing, ingestion, decomposition, audit/report regressions
│   │   ├── test_privacy.py                # Privacy/PHI behavior and session handling tests
│   │   ├── test_imports.py                # Import smoke tests (module availability)
│   │   ├── test_phi_protection.py         # Additional PHI / suppression tests
│   │   ├── test_deterministic_postprocessor.py  # Deterministic post-processor tests
│   │   ├── test_executive_output_formatter.py   # Executive formatter, exact-value fidelity, board language tests
│   │   ├── test_p6_forecast_builder.py    # Generic and surgical forecast builder tests
│   │   ├── test_p6_ed_forecast.py         # ED forecast builder/routing tests
│   │   ├── test_forecast_validator.py     # Forecast validator policy and interval tests
│   │   ├── test_forecast_output_schema.py # Forecast schema conformance tests
│   │   ├── test_sandbox_executor.py       # Sandbox timeout/safety and execution tests
│   │   ├── test_schema_forecasting_signature.py # Forecast-signature detection tests
│   │   └── __init__.py
│   │
│   └── (161 discovered test methods currently present across the test suite)
│
├── Logging & Audit
│   ├── logs/
│   │   └── audit.log.enc              # Encrypted persistent audit log written by audit_log.py
│   │
│   └── AuditAgent                     # Filesystem-pure in-memory audit record returned to UI badge formatter
│
├── Deployment & Documentation
│   ├── Dockerfile                     # Docker image build spec
│   ├── LICENSE                        # MIT License
│   ├── README.md                      # This file
│   ├── Healthcare_Augmentation_AI.code-workspace  # VS Code workspace config
│   ├── privacy_policy.md              # Privacy policy for users
│   ├── terms_of_service.md            # Terms of service
│   ├── internal_breach_protocol.md    # Data breach response procedures
│   ├── users.json                     # User store (if using file-based RBAC)
│   └── pyrightconfig.json             # Pylance type-checking config
│
└── Legacy (Not Currently Used)
    └── legacy/                        # Old analysis engines (preserved for reference)
        ├── healthcare_analysis.py
        ├── mdsi_analysis.py
        ├── scenario_planner.py
        ├── (etc.)
        └── README.md

Key Module Relationships

app.py (Gradio UI)
  ↓
services/analysis_service.py (top-level orchestration)
  ├→ services/input_preparation.py (file parsing + schema prep)
  ├→ services/analysis_execution_service.py (decomposition + routing + sandbox execution)
  ├→ services/pipeline_stages.py (validation/privacy stages)
  ├→ services/reporting_service.py (reporting + compliance + audit flow)
  ├→ validators/* (validation rules)
  └→ llm_router.py (LLM coordination)

llm_router.py
  ├→ circuit breaker (fault tolerance)
  ├→ rate limiter (request de-dup)
  └→ cohere.Client & local_llm (models)

🔧 Configuration & Environment Variables (Comprehensive Reference)

Critical Variables (Required)

Security & Compliance

LLM Configuration

Deployment & Networking

Privacy & De-identification

Advanced / Optional

Audit & Observability

Setting Environment Variables

Windows PowerShell:

$env:COHERE_API_KEY = "sk-xxxxxxxxxxxx"
$env:CLARITYOPS_ADMIN_PASSWORD = "MySecurePass123!"
$env:PHI_MODE = "true"
$env:PORT = "7860"

# Or load from .env file
Get-Content .env | ForEach-Object {
    if ($_ -match "^\s*([^=]+)=(.*)$") {
        $var = $matches[1].Trim(); $val = $matches[2].Trim()
        Set-Item -Path "env:$var" -Value $val
    }
}

macOS / Linux:

export COHERE_API_KEY="sk-xxxxxxxxxxxx"
export CLARITYOPS_ADMIN_PASSWORD="MySecurePass123!"
export PHI_MODE="true"
export PORT="7860"

# Or source from .env
set -a
source .env
set +a

Docker (via environment file):

FROM python:3.12
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .

# Use .env file or pass --env-file at runtime
CMD ["python", "app.py"]

Run with .env:

docker run --env-file .env -p 7860:7860 clarityops:latest

🤖 Customization & Extension Guide

Adding Custom Template Categories

ClarityOps can be extended to recognize new problem types without hardcoding. To add a custom template:

1. 1.Define Keywords in scenario_templates.py:

   _KW_CUSTOM_DOMAIN = {
       "keyword1", "keyword2", "keyword3", ...
   }

1. 1.Create a Specialized Builder in scenario_templates.py:

   def _build_custom_domain(dataframes: List[pd.DataFrame], fingerprints: List[SchemaFingerprint], ...) -> ScenarioMatch | None:
       # Return ScenarioMatch(code=...) if match found, else None
       return _generate_custom_analysis_code(...)

1. 1.Register in ScenarioEngine.try_match() prioritization chain:

   match = _build_custom_domain(dataframes, fingerprints, ...)
   if match: return match

1. 1.Test against your data:

   python -m pytest tests/test_routing_and_ingestion.py::YourCustomTest

Modifying Prompts & Policies

All LLM prompts and policy constraints are data-driven and easily customizable:

1. 1.LLM Prompts (in prompts/):
2. 2.system_master.txt — Main system prompt (controls tone, guardrails)
3. 3.system_two_phase.txt — Two-phase clarification prompt
4. 4.extractor_system.txt — For code generation phase
5. 5.Edit directly; changes take effect on next app restart

1. 1.Medical Guardrails (in policies/medical_guardrails.md):
2. 2.Document realistic staffing ratios, clinical outcome bounds, etc.
3. 3.Changes picked up by policy_validator.py at runtime
4. 4.Can be indexed via python build_policy_index.py for retrieval

1. 1.Global Constraints (in core/policy_global.json):
2. 2.Min/max staffing FTE, cost ranges, clinical outcome bounds
3. 3.Enforced by policy_validator.py on all outputs

Adding Custom Validators

New validation rules can be added without modifying core logic:

1. 1.Create a Validator Module in validators/custom_validator.py:

   def assert_valid(data: dict):
       # Your validation logic
       if not condition:
           raise ValueError(f"Custom validation failed: {reason}")

1. 1.Import in `analysis_service.py`:

   from validators.custom_validator import assert_valid as custom_assert_valid
   
   # In the validation loop:
   custom_assert_valid(validated_data)

1. 1.Test:

   python -m pytest tests/ -k custom_validator

Extending Privacy Controls

To add new privacy rules (e.g., masking clinic names, phone numbers):

1. 1.Add Regex Pattern in phi_protection.py:

   PHI_PATTERNS.append((re.compile(r"your_pattern"), "[REDACTED_CUSTOM]"))

1. 1.Or Add NER Rule (if using Presidio):

   # In compliance_agent.py
   custom_results = presidio_analyzer.analyze(text=content, language="en", ...)

1. 1.Test via test_privacy.py:

   python -m pytest tests/test_privacy.py -v

Integrating with External Systems

ClarityOps outputs can be piped to external systems:

1. 1.Audit Log Export:
2. 2.Persistent audit events are written by audit_log.py as encrypted records in ./logs/audit.log.enc (or the directory configured by CLARITYOPS_LOG_DIR)
3. 3.AuditAgent.run() returns an in-memory audit_record; it does not write audit files
4. 4.audit_log.py provides decryption, integrity-check, and export helpers for compliance review workflows

1. 1.API Integration (future):
2. 2.Analysis results in analysis_output.schema.json format
3. 3.Can be exported to EHR, data warehouse, reporting tool via custom export layer

1. 1.Custom Report Formats:
2. 2.Edit report_engine.py to add new report generators (PDF, Excel, JSON)
3. 3.Keep core analysis unchanged

💻 Usage Examples

Web UI Workflow

1. 1.Sign In: Use credentials set during setup_users.py
2. 2.Upload Files: Drag-and-drop or select CSV/XLSX/PDF files
3. 3.Ask Question: Type a healthcare operations or planning question
4. 4.Answer Clarifications: System asks clarification questions (Phase 1)
5. 5.Review Results: Final analysis with audit badge and compliance badge
6. 6.Export/Share: Copy results to clipboard or download as markdown

Example Prompts

• —"Which operating departments should we prioritize to reduce median wait times for outpatient visits?" → Triggers P0 Group-vs-Benchmark template
• —"How many additional FTEs would we need to screen all eligible patients over the next 3 months?" → Triggers P1 Operational template
• —"What is the cost per client for this diabetes prevention program, and how does it compare to industry benchmarks?" → Triggers P2 Financial template
• —"Calculate the staffing utilization rate across sites and flag any outliers." → Triggers P4 Utilization template

CLI / Pipeline Usage

For batch or programmatic analysis:

# Phase 1: Generate clarification questions
python -m pipeline.main \
  --scenario "Our clinic needs to optimize surgical capacity." \
  --name oncology_capacity_study

# User fills in clarifications.json interactively...

# Phase 2: Run full analysis
python -m pipeline.main \
  --pack packs/oncology_capacity_study \
  --phase2

# Results in: packs/oncology_capacity_study/analysis_output.json

Python SDK (Embedded Usage)

from services.analysis_service import AnalysisService

# Initialize service
service = AnalysisService()

# Run analysis against uploaded/local files
result_markdown = service.handle(
    "What is our capacity constraint?",
    files=["capacity.csv", "costs.csv"],
    yield_update=lambda _msg: None,
    phi_mode_active=False,
    request=None,
)

print(result_markdown)  # Final markdown report with audit/compliance sections

🔬 Advanced Topics

Scenario Template Matching Algorithm (Technical Deep Dive)

1. 1.Keyword Scoring (0.0–1.0):

   score = (matched_keywords_count) / (total_template_keywords)

• —Example: Query "capacity analysis for 3 teams" vs. P1 template
• —Matched: ["capacity", "teams"] = 2 keywords
• —P1 has ~30 keywords, so score = 2/30 ≈ 0.067
• —If threshold = 0.05, match succeeds

1. 1.Schema Fingerprinting:
2. 2.Each DataFrame scanned for domain indicators
3. 3."total_cost", "cost_usd" → Financial domain
4. 4."wait_time_days", "bed_occupancy" → Operational domain
5. 5."a1c_percent", "systolic_bp" → Clinical domain

1. 1.Specialized Builders:
2. 2.If keyword + schema both match, specialized builder invoked
3. 3.Builder generates deterministic Python code (high confidence)
4. 4.Code is audited independently, scored 0.90+

Audit Agent Self-Healing Mechanism (Production Pattern)

1. 1.Initial Run: Template or LLM generates code → sandbox executes → JSON validated
2. 2.Confidence score assigned (template: 0.90+, LLM: 0.50–0.80)

1. 1.First Self-Healing Attempt (if confidence < 0.95):
2. 2.LLM is asked: "This analysis scored {score}. Why? Fix issues."
3. 3.New code generated, executed, re-validated
4. 4.New confidence score assigned

1. 1.Best-Response Selection:
2. 2.Compare: rank = confidence + (0.2 if audit_pass else 0.0)
3. 3.Keep highest-ranking response; never downgrade
4. 4.If template output is 0.90, LLM self-heal attempt scores 0.70 → template retained

1. 1.Early Exit:
2. 2.If original output was deterministic AND high-confidence, skip self-healing entirely
3. 3.Preserves template determinism without unnecessary retries

Circuit Breaker State Machine (Fault Tolerance)

     CLOSED (normal)
        ↓
   [count failures]
        ↓
   [failures > 5?]
     Yes → OPEN
     No  → stay CLOSED
        ↓
     OPEN (outage)
        ↓
   [wait 60s]
        ↓
    HALF_OPEN (probe)
        ↓
   [try 1 request]
        ↓
   [success?]
     Yes → CLOSED (2 successes needed)
     No  → OPEN (wait 60s again)

� Deployment Options

Local Development

python app.py
# Launches on http://localhost:7860

Docker Deployment

docker build -t clarityops:latest .
docker run \
  --env-file .env \
  -p 7860:7860 \
  clarityops:latest

Hugging Face Spaces

This repo is already configured for Hugging Face Spaces:

• —sdk: gradio
• —app_file: app.py
• —sdk_version: 5.44.1

To deploy:

1. 1.Fork or create a new Space at huggingface.co/new-space
2. 2.Select "Gradio" and point to this repository
3. 3.Set secrets:
4. 4.COHERE_API_KEY
5. 5.CLARITYOPS_ADMIN_PASSWORD
6. 6.Space will auto-build and launch

Considerations for Spaces:

• —Free tier has memory/CPU limits; performance best for small files
• —Audit logs stored in ephemeral /tmp (cleared on restart)
• —For production privacy, use private Space + enterprise plan

Cloud Platforms (AWS ECS, Google Cloud Run, Azure Container Apps)

Dockerfile provided; deploy as containerized service:

• —Mount persistent volume for audit logs
• —Set environment variables via secrets manager (AWS Secrets, GCP Secret Manager)
• —Configure load balancer with SSL/TLS termination
• —RBAC + session management handles multi-tenant isolation

📊 Analysis Output Format & Results

ClarityOps returns fully structured, auditable analyses in a consistent format:

Analysis Output Schema

All analyses conform to schemas/analysis_output.schema.json:

{
  "analysis_id": "uuid",
  "timestamp": "ISO8601",
   "version": "4.0",
  "user_id": "analyst1",
  "query": "Original question",
  "generation_method": "scenario_template | llm_generated",
  
  "prioritization": {
    "groups_prioritized": ["Group A", "Group B"],
    "ranking_methodology": "descriptive",
    "confidence_score": 0.95,
    "rationale": "Group A has highest wait times..."
  },
  
  "capacity": {
    "formula_used": "teams × hours × utilization",
    "total_clients_capacity": 2880,
    "staffing_assumptions": ["6 teams", "40-hour weeks"],
    "constraints_identified": ["Budget limited", "Physical space constrained"]
  },
  
  "financial": {
    "cost_model": "cost_per_client × population_size",
    "startup_cost": 120000,
    "startup_per_client": 120,
    "ongoing_cost_annual": 450000,
    "ongoing_per_client": 45,
    "total_per_client": 165,
    "total_for_population": 39600,
    "benchmark_comparison": "Within industry range (10th–90th percentile)"
  },
  
  "clinical": {
    "primary_outcome": "A1C reduction",
    "expected_absolute_delta": -0.8,
    "pct_achieving_target": 0.42,
    "safety_considerations": "Monitor for hypoglycemia..."
  },
  
  "recommendations": [
    "Action 1: Allocate resources to highest-delay group",
    "Action 2: Standardize intake process",
    "Action 3: Track utilization metrics weekly"
  ],
  
  "data_sources_used": ["capacity.csv", "costs.xlsx"],
  "audit_badge": "✅ VERIFIED | Confidence: 0.95 | Latency: 1,240ms",
  "compliance_badge": "🔒 PASSED | PHIPA/PIPEDA compliant | Small-cell suppression: applied"
}

Markdown Report (User-Facing)

Reports are generated as formatted markdown for easy reading:

# Capacity Analysis Report
**Analysis ID**: abc123  
**Date**: 2026-04-05  
**Analyst**: John Doe  

## Executive Summary
This analysis evaluated capacity constraints for the Ontario regional clinic.

## Findings

### Operational Bottleneck
- **Constraint**: 3-month window supported only 2,880 client visits (60 working days × 48 visits/day)
- **Gap**: Current demand exceeds by 15%

### Team Staffing Implications
- **Available FTE**: 6 teams × 8 hours/day = 48 billable hours/day
- **Recommended Action**: +1.5 FTE to reach 90% occupancy target

## Financial Impact
- **Cost per client**: $165 (startup $120 + ongoing $45)
- **Total for 1,200 clients**: $198,000
- **Benchmark**: Aligns with industry median

## Clinical Outcomes (Diabetes Cohort)
- **Expected A1C reduction**: 0.8% absolute
- **Target achievement rate**: 42% reach A1C < 8%

✅ **Audit**: VERIFIED | Confidence: 0.95  
🔒 **Compliance**: PASSED | PHIPA/PIPEDA compliant

� Recent Improvements (April-May 2026)

This section documents critical fixes and enhancements completed to improve routing accuracy, audit reliability, code quality, and executive output standards.

🧾 Universal Uploaded-Data Recovery Guardrail (May 2026)

Decomposed sub-analyses now include an explicit guardrail that prevents false "missing data" outcomes when uploaded files contain usable reference metrics.

What changed:

• —Added deterministic normalization and metric lookup fallback in scenario_manager.py for key/value reference tables (e.g., Metric | Value | Source), including currency strings and simple ranges.
• —Hardened sub-analysis dataframe selection so reference tables are retained for relevant focused queries instead of being dropped by token-only scoring.
• —Added deterministic cost-summary recovery in decomposed runs: if a financial sub-analysis lacks structured cost output, the pipeline now derives startup, ongoing, per-client, and total screening cost from uploaded data before report generation.
• —Added note hygiene: stale missing-cost notes are removed when deterministic recovery succeeds.

Why it matters:

• —The system now uses uploaded data first, even when formatting is imperfect.
• —Financial sub-analyses no longer incorrectly classify present cost data as missing.
• —This fix is universal and path-level (decomposition guardrail), not scenario-specific.

Verification:

• —Targeted regression passed: test_financial_query_with_benchmark_comparison_still_uses_financial_template.
• —Syntax/type checks on updated decomposition path passed.

🧩 Gradio Runtime Compatibility (May 2026)

What changed:

• —Updated app wiring for runtime compatibility:
• —removed deprecated Chatbot constructor argument type="messages"
• —moved theme and css from gr.Blocks(...) to launch(...)
• —updated local bind behavior to use localhost-friendly launch defaults for direct browser access

Why it matters:

• —Prevents startup failures caused by incompatible constructor arguments.
• —Keeps local execution aligned with modern Gradio runtime behavior.

🏛️ Board-Ready Executive Output Standard & Data Fidelity (April 2026)

A new mandatory formatter layer — services/executive_output_formatter.py — now standardizes every final report for senior healthcare leadership use.

What changed:

• —Enforced a concise, decisive, hospital-board style across deterministic and LLM-backed reports
• —Added stronger action-oriented recommendations and a mandatory Board Recommendation box
• —Preserved exact benchmark and disparity values during final formatting to prevent numeric drift
• —Prioritized the largest validated equity gaps first using narrative directives and validated gap metadata
• —Added an explicit Indigenous Signal section when source workbooks contain Indigenous response evidence

Why it matters:

• —Outputs are immediately usable by a board, CEO, or clinical director
• —Executive polish no longer comes at the cost of data fidelity
• —Equity and Indigenous findings are made visible rather than buried in narrative

Verification:

• —Targeted formatter regressions pass, including idempotence and exact-value preservation checks
• —Verification references are maintained in-repo via tests/ and should be re-run in your target environment before release.

� P0 Routing Hardening (April 2026)

The latest hardening pass introduced an explicit query profile layer in services/routing_schema.py to classify incoming requests before analysis starts.

What changed:

• —Added build_query_profile() with route-family classification for simple, structured, strategic, and predictive requests
• —Updated Phase1Service to avoid unnecessary clarification prompts for clean workbook-based executive summaries
• —Updated scenario_manager.py so decomposition decisions now use both prompt wording and uploaded data context
• —Added a sheet-aware / join-aware data catalog in services/input_preparation.py to preserve workbook structure across multi-file analysis

Why it matters:

• —Riverside-style simple text prompts stay lightweight
• —Patient-experience workbook / board-report prompts now stay in single-analysis mode instead of being over-decomposed
• —Niagara-style option-comparison cases are more reliably identified as strategic multi-domain analyses
• —Predictive / messy-data cases are explicitly marked for higher-caution handling

�🐛 Regressions Fixed

1. Operational Capacity Window Parsing (scenario_templates.py)

Issue: Queries containing phrases like "3-month window" were incorrectly parsed as 5 days instead of the correct 60 working days (3 months × 20 business days/month). This caused capacity calculations to report 8× lower throughput than correct.

Root Cause: Regex pattern r"(\d[\d,]*)\s+months?\b" only matched whitespace-separated numeric + unit pairs, not hyphenated forms like "3-month window".

Fix: Added explicit regex alternatives in scenario_templates.py lines 862–864:

months = _extract_query_number(query, [
    r"(\d[\d,]*)\s+months?\b",           # "3 months", "3 month"
    r"(\d[\d,]*)\s*-\s*months?\b",       # "3-months"
    r"(\d[\d,]*)\s*-\s*month\s+window\b" # "3-month window"
])
working_days = explicit_days or (months * 20 if months else None)

Impact: Operational sub-analyses now correctly report 2,880–3,600 client capacity over 3-month periods.

Test: test_scenario_engine_operational_reference_table_projects_team_capacity ✅ verified.

2. Financial Template Routing (scenario_manager.py)

Issue: Financial questions incorporating benchmark-comparison clauses (e.g., "What is the screening cost compared to benchmarks?") were incorrectly rejected as "composite prompts" before deterministic builders could match them. This caused the system to fall back to LLM-only pathways or declare data as missing.

Root Cause: Composite-prompt detection threshold was too aggressive (if question_count >= 2) and ran before the specialized financial/operational/clinical builder checks. A single financial question + one comparison clause would trigger false rejection.

Fix: Reordered logic in scenario_manager.py lines ~340–365:

1. 1.First: Check specialized builders (_build_financial(), _build_operational(), etc.) using the focused sub-question
2. 2.Then: Apply composite-prompt guard only if no specialized match is found
3. 3.Raise threshold: question_count >= 3 (narrower rejection)

# Specialized builders get first chance
for specialized_builder in (..._build_financial, ...):
    match = specialized_builder(dataframes, fingerprints, query=query)
    if match: return match

# Then check composite threshold (now >=3 instead of >=2)
question_count = query.count("?")
if len(strong_matches) >= 2 and question_count >= 3: return None

Impact: Financial questions with benchmark comparisons now correctly route to deterministic template, calculating total_screening_cost = $1,062,000 from reference metric tables.

Test: test_financial_query_with_benchmark_comparison_still_uses_financial_template ✅ verified.

🛡️ Audit Layer Enhancements

3. Best-Response Preservation in Audit Loops (audit_agent.py)

Issue: The audit layer's self-healing mechanism occasionally degraded high-confidence deterministic responses. If a self-heal attempt scored lower than the original deterministic output (e.g., 0.0 vs. 0.9 confidence), the lower-scoring version was still accepted, effectively downgrading reliable answers.

Root Cause: The audit loop did not track the best-scoring response across all retry attempts—it only retained the "current" response in each iteration, allowing degradation when retry scores were poor.

Fix: Introduced rank-based best-response tracking in audit_agent.py lines 217–340:

best_response = primary_response
best_audit = None
best_rank = 0.0

for attempt_num in range(...):
    audit_result = self._audit(...)
    effective_rank = confidence + (0.2 if audit_pass else 0.0)
    
    # Keep the response with the highest effective rank
    if best_audit is None or effective_rank >= best_rank:
        best_rank = effective_rank
        best_audit = audit_result
        best_response = current_response
    
    # Early exit for deterministic outputs (preserve without self-heal)
    if generation_method in deterministic_methods and not high_severity_flags:
        break
        
return best_response, audit_record

Impact: Deterministic template responses (confidence 0.90) are now preserved when self-heal attempts score lower. The system maintains audit trail but does not blindly accept degraded rewrites.

Test: test_audit_agent_preserves_best_deterministic_response_when_self_heal_degrades ✅ verified.

✅ Code Quality & Type Safety

4. Pylance Type-Checking Cleanup (tests/test_routing_and_ingestion.py)

Issue: 17 Pylance type-checking errors were reported in the test suite:

• —4 errors on optional decomposition (subscript on potentially None object)
• —8+ errors on optional match.code (attribute access on potentially None)
• —2 errors on AuditAgent mock attributes (_score_history list vs. deque type mismatch)
• —3 errors on audit aggregate assignments (parameter name mismatches)

Root Cause: Type-narrowing helpers were missing for optional types returned by deterministic matching and decomposition functions.

Fix: Applied strategic enhancements in tests/test_routing_and_ingestion.py:

1. 1.Added imports:

   from collections import deque
   from typing import Any, cast

1. 1.Added type-narrowing helpers:

   @staticmethod
   def _require_match(match: Any) -> Any:
       assert match is not None
       return match
   
   @staticmethod
   def _require_decomposition(decomposition: Any) -> dict[str, Any]:
       assert decomposition is not None
       return cast(dict[str, Any], decomposition)

1. 1.Updated 5 test methods to use narrowing helpers instead of assertIsNotNone():

   # Before: assertIsNotNone(match)
   # After:
   match = self._require_match(ScenarioEngine().try_match(query, [df], ["file.csv"]))
   
   # Now Pylance recognizes match is not None ✓

1. 1.Fixed AuditAgent mock:
2. 2.Changed agent._score_history = [] → agent._score_history = deque()
3. 3.Changed agent._write_log = lambda record: → agent._write_log = lambda audit_record:

Impact: All 17 Pylance errors cleared; type checking now properly validates narrowed optional types.

Verification: get_errors on test file returns 0 errors ✅

🧪 Test Suite Verification

Current Test Surface: tests/ currently contains 161 discovered test methods ✅

Run locally to verify current status:

python -m pytest tests -q

Coverage includes:
  ✅ privacy/compliance protections
  ✅ import smoke checks
  ✅ routing, ingestion, template, and report regressions
  ✅ deterministic post-processing validation
  ✅ P0 route-profile and Phase 1 clarification hardening
  ✅ executive formatter structure, idempotence, exact-value fidelity, and board-language enforcement

Key Regression Tests:

• —test_scenario_engine_operational_reference_table_projects_team_capacity — 3-month window parsing ✅
• —test_financial_query_with_benchmark_comparison_still_uses_financial_template — Financial routing ✅
• —test_audit_agent_preserves_best_deterministic_response_when_self_heal_degrades — Audit preservation ✅
• —test_augmented_priority_sub_analysis_still_matches_priority_template — Template matching after augmentation ✅

🎯 Universal Applicability

All fixes are pattern-based and reusable across healthcare scenarios (not hardcoded to any specific case study):

• —Month/window parsing: Works for any N-month window, N-week period, or N-day span (numeric + time unit)
• —Deterministic routing: Uses schema fingerprinting + keyword scoring (adapts to any healthcare question type)
• —Audit best-response selection: Rank-based algorithm applies to any generation method (deterministic or LLM)
• —Type narrowing: Improves code maintainability for all future test additions

�🛣️ Roadmap

Based on the current codebase, likely next improvements include:

• —[ ] Additional P1 (Operational) templates: surgical capacity, ED throughput, clinic booking systems
• —[ ] Additional P2 (Financial) templates: multi-year financial models, budget variance analysis
• —[ ] Additional P3 (Workforce) templates: burnout risk scoring, turnover prediction
• —[ ] Real-world benchmark datasets (Ontario Health, CIHI) for automatic comparison
• —[ ] Template versioning and A/B testing framework
• —[ ] Direct connectors to healthcare databases (Epic, Cerner, provincial health systems)
• —[ ] Real-time data refresh (scheduled cache invalidation)
• —[ ] Data lineage tracking (audit trail from source system to output)
• —[ ] API gateway for programmatic access (REST + gRPC)
• —[ ] Forecast models (time-series extrapolation, demand planning)
• —[ ] Scenario simulation (what-if analyses with sensitivity tables)
• —[ ] HIPAA compliance mode (US deployments)
• —[ ] GDPR data residency controls
• —[ ] Mobile app (React Native) for results review
• —[ ] Performance dashboards (query latency, accuracy metrics, cost per analysis)
• —[ ] Multi-tenancy with resource quotas
• —[ ] LDAP/Active Directory integration (enterprise SSO)

🤝 Contributing

Contributions are welcome.

1. 1.Fork the repository
2. 2.Create a feature branch
3. 3.Make focused, well-documented changes
4. 4.Test locally
5. 5.Open a pull request describing the improvement and rationale

If you are changing prompts, policies, or validation logic, include a short example showing the expected behavior before and after the change.

📄 License

This project is distributed under the MIT License. See `LICENSE` for details.

🙏 Acknowledgments

ClarityOps builds on the work of several excellent tools and ecosystems:

• —Gradio for the application interface
• —Cohere for LLM capabilities
• —Hugging Face and Sentence Transformers for model hosting and embeddings
• —FAISS for lightweight retrieval indexing
• —Canadian privacy and de-identification guidance reflected in the project's policy and guardrail files

If you use this project in a regulated environment, make sure your deployment, access controls, and legal/privacy review align with your organization’s requirements.