DavidL72Code/UMB_Sustainable_Chatbot

Sustainable Labs ChatBot

A RAG (Retrieval-Augmented Generation) chatbot for the UMass Boston Sustainable Solutions Lab. Built by Team 1 "RAG's to Riches".

1. Why We Built This

The Sustainable Solutions Lab has information spread across project pages, staff profiles, annual reports, publications, and research summaries. A normal keyword search can find documents, but it does not reliably understand follow-up questions, pronouns, multiple facts in one question, or which source is authoritative.

We built this assistant to provide a conversational research interface that:

• —Answers questions about SSL using the lab's own corpus rather than general model knowledge.
• —Makes source-backed research, people, projects, and publications easier to explore.
• —Remembers enough recent conversation to resolve follow-ups such as “what did she study?”
• —Handles multi-part questions by separating their facets and preserving evidence for each facet.
• —Shows citations and diagnostic information so an answer can be reviewed instead of trusted blindly.

The design deliberately combines deterministic software with an LLM. Deterministic routing, source metadata, validation, and citation handling provide control and repeatability; the LLM handles language understanding, query rewriting, planning, and final composition where flexible language reasoning is useful.

2. Architecture

Three deployed pieces, split so the chat UI is never inside a Hugging Face iframe and the vector store ships with the backend image:

flowchart LR
    subgraph Browser["Browser"]
        UI["Chat UI + personal dashboard<br/>static HTML/CSS/JS"]
    end
    subgraph Vercel["Vercel · static host"]
        CDN["Static assets<br/>/api/* rewritten to the Space<br/>so cookies stay first-party"]
    end
    subgraph HF["Hugging Face Space · Docker"]
        API["Flask + SSE"]
        RAG["Retrieval + generation pipeline"]
        VS[("Chroma<br/>7,694 chunks · BGE 768-dim")]
        API --> RAG --> VS
    end
    subgraph Ext["External services"]
        GEM["Gemini API<br/>selector · generator"]
        SUP[("Supabase<br/>auth · visitor history · metrics")]
    end
    UI --> CDN --> API
    RAG --> GEM
    API --> SUP

3. How It Works

Every branch below exists because a specific class of question failed without it.

flowchart TB
    Q(["User question"]) --> G{"Safety and<br/>rate limit"}
    G -->|blocked| X(["Refusal<br/>nothing retrieved"])
    G -->|ok| ST["<b>Conversation state</b><br/>resolve pronouns against the active subject"]
    ST --> LR["<b>Local router</b> — always runs<br/>classifies the question and scopes it<br/>from the entity and document registries"]

    LR --> ROUTE["<b>Query route</b><br/>scope · question type · facets"]

    ROUTE --> F{"Evidence<br/>from where?"}
    F -->|"a registry row"| EX["<b>Deterministic extractor</b><br/>staff rows, contacts, field lookups<br/><i>answer composed in code</i>"]
    F -->|"the document corpus"| RET["<b>Hybrid retrieval</b><br/>dense + BM25 + rare-term, per facet<br/><i>detailed below</i>"]

    RET --> SEL["<b>Evidence selector</b> (1 call)<br/>pick the answer-bearing blocks<br/>out of ~28 candidates"]
    SEL --> GEN["<b>Generation</b> (1 call)<br/>evidence-only prompt<br/>greedy decode, fixed seed"]

    EX -->|"no generation call"| VAL["<b>Validation</b><br/>numbers · contract · citations"]
    GEN -->|"written by the model"| VAL
    VAL --> OUT(["SSE to the browser<br/>allowlisted fields only"])

Inside hybrid retrieval. Three retrievers cover each other's blind spots, then the candidate set is narrowed without letting one document dominate.

flowchart LR
    IN(["Facet query"]) --> D["Dense<br/>BGE 768-dim<br/><i>paraphrase, concept</i>"]
    IN --> B["BM25<br/><i>names, acronyms, titles</i>"]
    IN --> RP["Rare-term passage<br/><i>one sentence buried<br/>in 600 words</i>"]
    D --> FU["RRF<br/>fusion"]
    B --> FU
    RP --> FU
    FU --> RR["Rerank<br/><i>metadata, freshness,<br/>route boost</i>"]
    RR --> DD["Dedupe<br/><i>only if candidate is a<br/>subset of what is kept</i>"]
    DD --> SD["Diverse seeds<br/><i>per-document cap</i>"]
    SD --> NB["Neighbour expansion<br/><i>adjacent chunks</i>"]
    NB --> OUT(["~28 candidates"])

The validation gauntlet. Every draft passes five checks before it ships. Each one exists because a specific wrong answer got through without it.

What each step does, and why it is there

4. Features

The chatbot answers questions about SSL research projects, publications, staff, initiatives, funding, and community partnerships using only the lab's own source documents. Everything the model says is grounded in retrieved chunks — no free-form invention.

User-Facing Features

• —Grounded answers drawn directly from SSL source documents (annual reports, project pages, publications, staff bios).
• —Streaming responses — text appears token by token as Gemini generates it, using Server-Sent Events.
• —Suggested questions — starter buttons on first load plus verified follow-up chips after some answers.
• —Saved sessions sidebar — one row per session, titled with the message that opened it. Clicking a row reopens that session and continues it; + New starts a fresh one, and deleting asks first. Signed-in visitors keep their sessions across logins, capped at 200 saved messages each.
• —Content filter — blocks profanity, hate speech, threats, and SSL/UMB-targeted harassment with a custom whitelist for legitimate academic terms (e.g. assessment, massachusetts, bird species) and a custom block list for org-specific phrases.
• —Friendly error handling — Gemini 503/429 errors surface as "high demand, try again" instead of raw stack traces.
• —Citation-aware answers — citations are normalized against the final answer and filtered to sources actually shown to the user.
• —Personal analytics dashboard at /dashboard, open without a login and scoped to the caller's own activity: latency, tokens, cost, retrieval path, cited sources, corpus coverage, and low-confidence cases. Anonymous visitors see the current session only; signed-in visitors also see their saved chats. The aggregate staff view over every visitor's chats stays behind an admin session.
• —Optional visitor accounts — signing in only controls whether a visitor's own history is saved; answers are identical either way.

Document Ingestion

At first run, `SEED_DOCUMENTS/` is parsed into structured units:

• —Project pages get split per project (`split_project_sections`).
• —Staff/board/affiliate pages get split per person with name detection (`split_people_sections`).
• —Slide decks get split per slide (`split_slide_sections`).
• —Everything else is chunked with RecursiveCharacterTextSplitter.

Each chunk is embedded and stored in ChromaDB with rich metadata (title, category, folder, source path, section name, chunk level). The metadata is what makes routing and reranking possible.

5. Models and Cost per Answer

Two model calls per answer. The work is split across two tiers so the expensive model only does what needs it. A third LLM planning stage exists in the code but is disabled — enabling it regressed 4 of 50 benchmark questions and fixed none.

Published paid-tier rates, USD per 1M tokens (pricing, checked 2026-09-05). Thinking tokens bill at the output rate:

A typical answer runs roughly 9k input and 750 output tokens across the two calls, which lands around $0.004 per answer — about 250 questions per dollar. The dashboard reports the real figure per answer rather than an estimate, computed from the token counts the API returns.

6. Security Model

A deliberate exception for this demo

The employee dashboard access control is fully implemented — and switched off for the public demo on purpose.

The staff dashboard aggregates every visitor's chats, and that aggregate view is still gated: /api/dashboard, /api/dashboard/interaction/<id> and their pages all require an admin session, exactly as built. make_admin_users.py, the Supabase staff role, and the fail-closed auth path are all in place and documented above.

What the demo does instead is expose a separate personal dashboard that carries the same operational depth — latency, tokens, cost, retrieval path, cited sources, corpus coverage, low-confidence cases — but scoped to whoever is looking. That way a reviewer can see how the observability works without being handed admin credentials, and without anyone's conversations being published.

Turning the staff view back on for a real deployment is configuration, not code: set DASHBOARD_SESSION_SECRET plus either ADMIN_USERS_JSON or a Supabase staff role, and sign in at /admin/login.

7. Tech Stack

Backend — Python 3 · Flask (REST + SSE) · Google Gemini (google-genai) · ChromaDB · sentence-transformers (BAAI/bge-base-en-v1.5) · custom BM25 · langchain-text-splitters · pypdf · better-profanity · python-dotenv

Frontend — HTML / CSS / vanilla JS, no framework · Server-Sent Events · fetch + ReadableStream · client-side Markdown rendering

Auth and data — Supabase Auth and Postgres, row-level security

Hosting — Hugging Face Spaces (Docker) · Vercel (static frontend, /api proxy)

8. Evaluation

Tuning ran against targeted failure subsets, not aggregate scores, with chunk-level tracing on every run so we could see which chunks retrieval returned and which stage lost the answer. Larger subsets and full runs then showed where overall performance actually stood. Every failure we fixed turned out to be a structural bug — evidence mangled before the prompt, a dedupe that deleted the longer chunk, a validator misreading 2020-21 as an invented number, a router matching project inside projected — not a tuning gap.

Two 208-question sets, scored by a separate Gemini judge pass on correctness against the corpus, citations, hallucination, and whether every part of the question was answered. The scores count correctness failures; citation-only mismatches, where the answer is right but cites a different valid source than the question expected, are listed separately.

Every remaining failure is a judge disagreement, each checked against the source by hand:

• —fs_111 — the per-group figures 88%, 87%, 86% are on page 32 of Views that Matter, under "has probably been happening".
• —n146 — Table 7.9's asset values are on page 134 of the cited report, and the same numbers passed in the previous run.
• —n156 — Figure 4 lists its four labels then its four value rows in the same order, giving Black 27%; the judge swapped Black and Latino/a.
• —n199 — the answer is right but drawn from a different valid document than the expected reference.

The first three share one cause: the judge is handed a corpus excerpt that does not contain the passage the answer came from, so a correct answer scores 1 out of 5. That is a limitation of the harness, not of the pipeline, which has no known unfixed defect on either set.

The last one fixed was n168, which asked for Boston's projected annualized flood losses and answered that the documents did not state them. They were in an indexed chunk of the target publication all along: the router matched project inside projected, scoped the question to the projects registry, and the answer chunk was capped out of the pool for source diversity. Word-boundary matching fixed it with no measured cost — 416 questions re-run, one newly passing, no regressions.

Generation is greedy with a fixed seed, but the evidence selector is a separate model call that can fall back on a 503, so a single run moves by about ±1 question. Compare full runs, not individual questions.

9. Running It

pip install -r requirements.txt
export GEMINI_API_KEY=your-key
python3 Chatbot.py            # http://localhost:7860

First run builds the vector store from SEED_DOCUMENTS/; after that it loads the committed Chroma index.

Deployed as a Hugging Face Space (Docker, backend + vector store) with the static frontend on Vercel, which proxies /api to the Space so session cookies stay first-party. Environment variables, the Supabase schema, staff accounts and the full deploy steps are in [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md).