dmissoh/provita-ai-bot

Kaddi.ai for Provita Clinic

Prototype of an administrative support assistant for Provita Clinic. The assistant answers approved patient questions, captures booking intent, collects feedback, and escalates safely to staff. WhatsApp transport is deliberately deferred until the agent core, retrieval, guardrails, and tools are validated end to end.

The YAML frontmatter above is consumed by Hugging Face Spaces when this repo is pushed as a Docker Space; it is harmless on GitHub. See the Deployment section below.

Status

Working PoC. Agent core, RAG, guardrails, tool calling, CLI, FastAPI service with static chat UI, evaluation harness (31/31), and Docker image are all in. Live demo runs in a browser via the FastAPI app; users can toggle between Groq and OpenRouter providers.

Live demo

• —URL: https://dmissoh-provita-ai-bot.hf.space
• —Hugging Face Space: https://huggingface.co/spaces/dmissoh/provita-ai-bot
• —GitHub repo: https://github.com/CompareIN/provita-ai-bot

The demo is hosted as a public Hugging Face Space (Docker SDK, free CPU tier). Two providers are wired and selectable via the pill toggle in the header:

• —openrouter · google/gemini-2.5-flash — default, eval baseline 31/31
• —groq · llama-3.3-70b-versatile — faster alternative, free tier with a daily token cap

Switching providers starts a fresh conversation. Nothing the assistant captures is real: bookings and escalations land in an audit log inside the container, which resets when the Space restarts.

A test script for stakeholders is in PROGRESS.md (16 representative questions covering FAQ, booking, insurance, doctor availability, medical advice, emergency, pricing, out of scope, and multi turn).

What this project is

A narrow operational PoC. Not a booking automation, not a medical assistant, not a mobile app replacement.

The assistant must:

• —answer approved administrative questions (clinic info, services, diagnostics, insurance partnerships, facility, appointment process)
• —cite the source files used for grounded answers
• —refuse medical advice and emergencies, and route them to humans or emergency services
• —capture booking intent without confirming or changing appointments
• —summarize cases for staff handoff
• —log enough information for audit and review

Repository layout

.
├── README.md                     This file (live demo URL, run + deploy guide)
├── PROGRESS.md                   Status mirror + 16 partner-facing test questions
├── Dockerfile                    HF Spaces image
├── compose.yaml                  Local docker compose entry
├── .env.example                  Provider keys + paths (copy to .env)
├── pyproject.toml
├── requirements.txt
├── app/
│   ├── adapters/                 Transport adapters (CLI, FastAPI + static UI)
│   ├── agent/                    Core: prompts, guardrails, agent loop
│   ├── providers/                ModelProvider (OpenRouter, Groq, Anthropic, MLX)
│   ├── rag/                      LlamaIndex + Chroma ingest and retrieval
│   ├── tools/                    capture_booking_intent, escalate_to_staff, ...
│   ├── storage/                  SQLite (conversations, turns, bookings, etc.)
│   └── ui/                       Static HTML chat UI served by FastAPI
├── eval/
│   ├── cases/scenarios.py        25 scripted Provita-style sessions
│   ├── harness.py                Eval runner + report writer
│   └── reports/                  Generated markdown reports per run
├── provita-rag-input/            Clinic knowledge corpus (RAG source)
│   ├── README.md                 Corpus overview, source URLs, coverage gaps
│   ├── clinic-profile.md
│   ├── services.md
│   ├── diagnostics-technology.md
│   ├── insurance-partnerships.md
│   ├── facility-infrastructure.md
│   ├── appointment-booking.md
│   └── assistant-boundaries.md   Guardrails text used in the system prompt
├── tests/                        Smoke + unit tests (pytest)
└── data/                         Runtime: Chroma index, SQLite db (gitignored)

AGENTS.md and .assisted/ are local working notes for AI agents and are gitignored.

Architecture (MVP)

Transport-agnostic agent core, designed so the CLI / web prototype can later be served behind a WhatsApp adapter without changing the agent. Legend: [X] = built and verified end to end, ( ) = not built yet.

  Patient on phone / browser                    Clinic staff (Slack channel)
            |                                            ^
            v                                            |
+-----------+-----------------+              +-----------+-----------------+
| Public host: HF Space [X]   |              | Staff hand-off              |
|  dmissoh-provita-ai-bot     |              | Slack incoming webhook [X]  |
|  .hf.space (Docker, free)   |              |  - escalate_to_staff posts  |
+-----------+-----------------+              |  - collect_feedback posts   |
            |                                +-----------+-----------------+
            v                                            ^
   CLI [X]      Static HTML chat UI [X]      WhatsApp adapter ( )
       \                |                           /
        \      transport adapters                  /
         \              |                         /
          v             v                        v
         +-------------------------------------+
         |       FastAPI Chat Service  [X]     |
         |   /providers /conversations /chat   |
         |   /turns  /admin (basic auth) [X]   |
         |   /  /webhooks/whatsapp ( )         |
         +-----------------+-------------------+
                           |
                           v
         +-------------------------------------+
         |             Agent Core              |
         |   system prompt + guards     [X]    |
         |   retrieval injection        [X]    |
         |   source citation logic      [X]    |
         |   conversation history       [X]    |
         |   hard-rule guardrails       [X]    |
         |    + emergency bypass        [X]    |
         |    + medical/pricing/insurer [X]    |
         |   tool router (loop)         [X]    |
         |   intent classifier          ( )    |
         +-+----------------+--------------+---+
           |                |              |
   +-------+                |              +-----------+
   |                        |                          |
   v                        v                          v
+-------------+   +----------------------+  +-------------------+
| Model Layer |   | Tool Layer       [X] |  |  Retrieval   [X]  |
+-------------+   +----------------------+  +-------------------+
| OpenRouter  |   | capture_booking      |  | LlamaIndex        |
|  gemini-2.5-|   |  _intent       [X]   |  |  + Chroma         |
|  flash [X]  |   | escalate_to_staff[X] |  | bge-small-en      |
| Groq        |   |  -> Slack notify [X] |  | top-k chunks      |
|  llama-3.3- |   | collect_feedback [X] |  | with source ids   |
|  70b [X]    |   |  -> Slack notify [X] |  +---------+---------+
| Anthropic   |   | check_doctor_        |            |
|  (stub)     |   |  availability  ( )   |            v
| MLX (stub)  |   +-----------+----------+  +-------------------+
+------+------+               |             |  Knowledge Base   |
       |                      v             |  provita-rag-     |
       |          +------------------------+|   input/ [X]      |
       |          |   Storage (SQLite)     |+---------+---------+
       |          +------------------------+          |
       |          | conversations    [X]   |          |
       +--------->| turns            [X]   |          |
                  | bookings         [X]   |<---------+ ingest
                  | feedback         [X]   |
                  | escalations      [X]   |
                  | JSON transcripts ( )   |
                  +------------------------+

Architecture (Diagram)

[image]

<!-- Source: architecture.mmd

flowchart TB
    classDef done fill:#d8ebe5,stroke:#36a191,color:#0f4d44
    classDef todo fill:#fef3c7,stroke:#b45309,color:#92400e,stroke-dasharray:4 3
    classDef ext  fill:#f1f5f9,stroke:#475569,color:#0f172a

    Patient([Patient on phone or browser])
    Staff([Clinic staff in Slack channel])

    subgraph Host["Public host"]
      HF["HF Space dmissoh-provita-ai-bot Docker free tier"]:::done
    end

    subgraph Adapters["Transport adapters"]
      direction LR
      CLI["CLI"]:::done
      UI["Static HTML chat UI Provita brand"]:::done
      WA["WhatsApp adapter"]:::todo
    end

    subgraph API["FastAPI Chat Service"]
      Routes["/providers, /conversations, /chat, /turns, /<br/>/admin (basic auth)<br/>/webhooks/whatsapp todo"]:::done
    end

    subgraph Agent["Agent Core"]
      SP["system prompt and guardrails verbatim"]:::done
      RAG["retrieval injection"]:::done
      CITE["source citation logic"]:::done
      HIST["conversation history"]:::done
      GR["hard-rule guardrails<br/>emergency bypass + runtime guidance"]:::done
      LOOP["tool router loop"]:::done
      IC["intent classifier"]:::todo
    end

    subgraph Models["Model Layer"]
      OR["OpenRouter gemini-2.5-flash default"]:::done
      GQ["Groq llama-3.3-70b-versatile"]:::done
      ANT["Anthropic stub"]:::ext
      MLX["MLX stub Qwen 3 / Gemma 3"]:::ext
    end

    subgraph Tools["Tool Layer"]
      BK["capture_booking_intent"]:::done
      ES["escalate_to_staff"]:::done
      FB["collect_feedback"]:::done
      DA["check_doctor_availability"]:::todo
    end

    subgraph Retrieval["Retrieval"]
      LI["LlamaIndex + Chroma<br/>bge-small-en-v1.5 embeddings"]:::done
      KB[("Knowledge base<br/>provita-rag-input/")]:::done
    end

    subgraph Notify["Staff hand-off and review"]
      SLACK["Slack incoming webhook<br/>STAFF_SLACK_WEBHOOK_URL"]:::done
    end

    subgraph Storage["SQLite audit log /data/provita.db"]
      DB[("conversations, turns,<br/>bookings, feedback, escalations")]:::done
    end

    Patient --> HF --> Adapters --> API --> Agent
    Agent --> Models
    Agent --> Tools
    Agent --> Retrieval
    Retrieval -.- LI -.- KB
    Tools --> DB
    Agent --> DB
    ES --> SLACK
    FB --> SLACK
    SLACK --> Staff

-->

Key principles:

• —The agent core is unaware of which transport is serving it (CLI, FastAPI, future WhatsApp).
• —The model layer is hidden behind a provider interface. OpenRouter (google/gemini-2.5-flash) is the default and the eval baseline (31/31). Groq (llama-3.3-70b-versatile) is a faster alternative behind a UI toggle, useful when its free-tier daily token cap is fresh. Anthropic and MLX are swap-in alternatives. The chat UI exposes a per-session toggle so a stakeholder can flip providers mid-demo.
• —Tools are pure functions over SQLite.
• —Retrieval is vector RAG (LlamaIndex with Chroma) from day one, with source citations enforced on every grounded answer.
• —Every turn is persisted to SQLite: user message, retrieved chunks, model answer, cited sources, tool calls, and the model id used.

Tech stack

• —Python 3.12 with FastAPI + Uvicorn
• —OpenRouter (default for the live demo, google/gemini-2.5-flash, eval baseline 31/31)
• —Groq (faster alternative behind UI toggle, llama-3.3-70b-versatile, free tier with daily cap)
• —LlamaIndex with Chroma + BAAI/bge-small-en-v1.5 embeddings for retrieval, source citations enforced
• —SQLite for persistence (conversations, turns, bookings, feedback, escalations)
• —Three agent tools: capture_booking_intent, escalate_to_staff, collect_feedback
• —Optional Slack incoming-webhook for real-time staff hand-off and review notifications
• —Provita-branded mobile-responsive HTML+JS chat UI (Montserrat, teal/sage palette, no build step, no framework)
• —Read-only admin dashboard at /admin (basic auth) for triaging the audit log
• —Docker image + compose.yaml targeting Hugging Face Spaces for the public demo
• —MLX with Qwen 3 or Gemma 3 as a later local swap option

Evaluation status

The eval harness in eval/ exercises 31 scripted scenarios across 10 categories (FAQ, booking, insurance, doctor availability, medical advice, emergency, pricing, out of scope, multi-turn, feedback). The current baseline against google/gemini-2.5-flash is 31/31 passing. Reports are written to eval/reports/.

python -m eval.harness --report eval/reports/run.md

How to run

Prerequisites

• —Python 3.12 (LlamaIndex and Chroma are not yet on 3.13+)
• —At least one LLM API key:
• —OpenRouter (default, eval baseline at 31/31): https://openrouter.ai/keys
• —Groq (faster alternative behind the UI toggle, free tier with a daily token cap): https://console.groq.com/keys
• —or Anthropic direct, if you already have an Anthropic key

If multiple keys are set, the auto-detect order is openrouter then groq then anthropic. The /providers endpoint returns whichever providers are configured, and the chat UI shows a toggle so a stakeholder can flip mid-demo.

Install

python3.12 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
cp .env.example .env
# edit .env and set either OPENROUTER_API_KEY or ANTHROPIC_API_KEY

The provider is auto-detected from whichever key is set. To force a choice, set PROVIDER=openrouter (or anthropic, mlx) in .env.

Verify scaffolding

pytest -q

Sanity check the LLM connection

python -m app.providers.ping
# or with a custom prompt
python -m app.providers.ping "What model are you?"

Run

# Ingest the knowledge base into Chroma (one-off; idempotent, --reset to rebuild)
python -m app.rag.ingest

# CLI chat client
python -m app.adapters.cli

# FastAPI + static chat UI (open http://localhost:8000)
uvicorn app.adapters.api:app --reload

# Admin dashboard (set ADMIN_BASIC_AUTH first; opens http://localhost:8000/admin)
ADMIN_BASIC_AUTH=admin:devpass uvicorn app.adapters.api:app --reload

# Evaluation harness
python -m eval.harness                                          # all 31 cases against the active provider
python -m eval.harness --category emergency                     # one category
python -m eval.harness --category feedback                      # one category (5 cases)
python -m eval.harness --case faq-services                      # one case
python -m eval.harness --quick                                  # 1 per category
python -m eval.harness --report eval/reports/run.md             # write a markdown report
PROVIDER=groq python -m eval.harness --report eval/reports/groq.md  # force a provider for one run

HTTP endpoints

Deployment

The app is packaged as a Docker image targeting Hugging Face Spaces (free, public, supports the full Python stack including the local embedding model).

Build and run locally

The recommended path is Docker Compose (compose.yaml in the repo root). It picks up .env automatically, mounts a named volume for /data so the Chroma index, SQLite audit log, and HuggingFace model cache survive restarts, and gives you a one-line up/down workflow.

docker compose up --build       # first run: builds the image, then boots the app
docker compose up               # subsequent runs: just boot
docker compose logs -f          # tail logs
docker compose down             # stop and remove the container
docker compose down -v          # also nuke the data volume for a clean slate
# open http://localhost:7860

If you prefer plain docker, the equivalent is:

docker build -t kaddiai .
docker run --rm -p 7860:7860 --env-file .env kaddiai

The container ingests the knowledge base on boot, then starts uvicorn on port 7860 (HF Spaces convention). The /data directory inside the container holds the Chroma index, SQLite db, and HuggingFace model cache.

Deploying a change to the live Space (day to day)

Once the Space exists and secrets are set, shipping a change is a three step loop. The HF Space rebuilds itself automatically on every push to its main.

# 1. make your change locally, then verify
source venv/bin/activate
pytest -q                                   # unit tests
python -m eval.harness --quick              # one case per category, smoke level
docker compose up --build                   # browser test at http://localhost:7860

# 2. commit
git add <files>
git commit -m "td-9f2974: <type>: <short description>"

# 3. push to both remotes
git push origin main                        # GitHub source of truth
git push hf main                            # triggers HF Space rebuild

After the HF push, the Space build takes 5 to 10 minutes for a full rebuild (longer when requirements.txt changes). Watch progress at https://huggingface.co/spaces/dmissoh/provita-ai-bot or via the API:

curl -s https://huggingface.co/api/spaces/dmissoh/provita-ai-bot \
  | python -c "import sys,json;r=json.load(sys.stdin)['runtime'];print('stage:',r.get('stage'),'  err:',r.get('errorMessage'))"

Stages you'll see in order: BUILDING → RUNNING_BUILDING → RUNNING. On failure: BUILD_ERROR or RUNTIME_ERROR with a message; check the Logs tab on the Space page.

End to end smoke test of the live demo after a deploy:

curl -s https://dmissoh-provita-ai-bot.hf.space/health
curl -s https://dmissoh-provita-ai-bot.hf.space/providers

/health should return {"status":"ok"} and /providers should list both providers.

Adjusting Space configuration

Two surfaces:

• —Code, Dockerfile, prompts, corpus: change the file, commit, push to hf. The Space rebuilds.
• —Secrets and env vars: edit at https://huggingface.co/spaces/dmissoh/provita-ai-bot/settings under "Variables and secrets". Adding or changing a secret restarts the Space without a rebuild. Use Secrets (not Variables) for OPENROUTER_API_KEY, GROQ_API_KEY, STAFF_SLACK_WEBHOOK_URL, and ADMIN_BASIC_AUTH so they aren't visible after creation.

Staff hand-off and feedback review

When escalate_to_staff or collect_feedback fires, a row lands in the SQLite audit log. There are two ways to surface this to staff:

Real-time push: Slack webhook. Set STAFF_SLACK_WEBHOOK_URL (create one at https://api.slack.com/messaging/webhooks). Both surfaces share the same webhook:

• —Escalations post as *Provita escalation* [HIGH] reason: ... with conversation id, summary, and recommended action.
• —Feedback posts as *Provita feedback* [4/5] with conversation id, rating, comment, visit context, and patient name when available. For ratings of 3 or below (or unrated complaints), the message also includes a Contact: line with the callback phone or email if the patient shared one.

Triage view: admin dashboard at `/admin`. Set ADMIN_BASIC_AUTH=username:password (use a HF Space Secret in production). The dashboard has three tabs over the audit log:

• —Feedback: timestamp, star rating, patient, comment, callback contact, visit context, conv id.
• —Escalations: timestamp, urgency badge (color-coded high/medium/low), reason, summary, recommended action, conv id.
• —Bookings: timestamp, patient, phone, email, specialty, preferred date/time, consultation type, payment, conv id.

Each row has a status dropdown (open / in_progress / resolved) plus an outcome-note field. Saving fires PATCH /admin/api/<entity>/{id}, records the basic-auth username as last_updated_by along with a timestamp, and shows "updated by ... at ..." beneath the controls. State persists in SQLite; status changes do not push to Slack.

Pick the row count (25 / 50 / 100 / 250). All three tabs are fetched lazily and refresh on demand. Returns 503 when ADMIN_BASIC_AUTH isn't configured, 401 on bad credentials, 200 with {rows, count} JSON on the API endpoints. PATCH endpoints validate status against the allowed values and return 422 on unknown values, 404 when the row id does not exist.

Without the webhook or admin creds configured, everything still works; staff just have to query feedback, escalations, and bookings tables manually with sqlite3 data/provita.db ....

Initial Space setup (one time, already done)

Recorded for reference. The Space dmissoh/provita-ai-bot was created with the hf CLI:

hf auth login
hf repos create dmissoh/provita-ai-bot --type space --space-sdk docker --public --flavor cpu-basic
git remote add hf git@hf.co:spaces/dmissoh/provita-ai-bot
# first push needed --allow-unrelated-histories because HF auto creates README.md
git pull hf main --allow-unrelated-histories -X ours
git push hf main
# then add OPENROUTER_API_KEY and GROQ_API_KEY as Secrets in the Space settings

The HF Space accepts SSH only when your public key is registered at https://huggingface.co/settings/keys. The local SSH config has a Host hf.co block pinning the right IdentityFile.

Storage on HF free tier

The free tier wipes /data on every restart. The corpus is small enough that boot time ingest takes only a few seconds. For persistent SQLite audit logs across restarts, attach a paid persistent storage volume; the existing DB_PATH=/data/provita.db will survive.

Alternative: Fly.io (private demo)

If Provita prefers a private URL, deploy the same Dockerfile to Fly.io with a fly.toml and a 1GB volume mounted at /data. Add HTTP basic auth at the FastAPI layer if needed. Pricing is roughly $3 to $5 per month for shared-cpu-1x.

Inspecting persisted conversations

Every CLI / API turn is logged to data/provita.db (SQLite). Three ways to read it:

1. Admin dashboard (recommended for staff). With ADMIN_BASIC_AUTH set, open /admin for a triage view of feedback, escalations, and bookings with filters and color-coded urgency.

2. Slack channel. With STAFF_SLACK_WEBHOOK_URL set, every escalation and feedback row also pushes a notification in real time.

3. Direct SQL (engineering / debugging):

sqlite3 data/provita.db ".tables"
sqlite3 data/provita.db "SELECT id, started_at, transport FROM conversations ORDER BY started_at DESC LIMIT 5;"
sqlite3 data/provita.db "SELECT seq, user_msg, cited_sources_json FROM turns ORDER BY ts DESC LIMIT 10;"
sqlite3 data/provita.db "SELECT created_at, urgency_flag, reason, substr(summary,1,80) FROM escalations ORDER BY created_at DESC LIMIT 10;"
sqlite3 data/provita.db "SELECT created_at, rating, substr(comment,1,80) FROM feedback ORDER BY created_at DESC LIMIT 10;"

Rows include the retrieved chunk ids, cited sources, tool calls, and model id used for each turn, so you can replay or audit any conversation later.

Task management

Tasks are tracked in td. The MVP epic is td-9f2974.

Useful commands:

• —td next highest priority open task
• —td tree td-9f2974 full epic tree
• —td start <id> begin work
• —td list --json machine readable

Out of scope (for now)

• —WhatsApp transport (Twilio or 360dialog)
• —Medesk integration of any kind
• —Autonomous booking confirmation, cancellation, or rescheduling
• —Medical advice, triage, or diagnosis
• —A native mobile application
• —Real time doctor availability, until a verified data source is provided
• —Specific insurer eligibility and pricing answers, until a verified list is provided

References

• —Progress and test questions: PROGRESS.md (todo mirror of td + 16 partner-facing test scenarios)
• —Source corpus overview: provita-rag-input/README.md
• —Live demo: https://dmissoh-provita-ai-bot.hf.space
• —HF Space: https://huggingface.co/spaces/dmissoh/provita-ai-bot
• —GitHub repo: https://github.com/CompareIN/provita-ai-bot

Internal scoping documents (working spec, council feedback, effort estimate) live in .assisted/ locally and are not committed.