DavidL72Code/UMB_Sustainable_Chatbot

1---2title: SSL Research Assistant3emoji: 🌱4colorFrom: blue5colorTo: yellow6sdk: docker7app_port: 78608pinned: false9---10 11# Sustainable Labs ChatBot12 13A RAG (Retrieval-Augmented Generation) chatbot for the UMass Boston Sustainable Solutions Lab. Built by Team 1 "RAG's to Riches".14 15## 1. Why We Built This16 17The Sustainable Solutions Lab has information spread across project pages, staff profiles, annual reports, publications, and research summaries. A normal keyword search can find documents, but it does not reliably understand follow-up questions, pronouns, multiple facts in one question, or which source is authoritative.18 19We built this assistant to provide a conversational research interface that:20 21- Answers questions about SSL using the lab's own corpus rather than general model knowledge.22- Makes source-backed research, people, projects, and publications easier to explore.23- Remembers enough recent conversation to resolve follow-ups such as “what did she study?”24- Handles multi-part questions by separating their facets and preserving evidence for each facet.25- Shows citations and diagnostic information so an answer can be reviewed instead of trusted blindly.26 27The design deliberately combines deterministic software with an LLM. Deterministic routing, source metadata, validation, and citation handling provide control and repeatability; the LLM handles language understanding, query rewriting, planning, and final composition where flexible language reasoning is useful.28 29## 2. Architecture30 31Three deployed pieces, split so the chat UI is never inside a Hugging Face iframe32and the vector store ships with the backend image:33 34```mermaid35flowchart LR36    subgraph Browser["Browser"]37        UI["Chat UI + personal dashboard<br/>static HTML/CSS/JS"]38    end39    subgraph Vercel["Vercel · static host"]40        CDN["Static assets<br/>/api/* rewritten to the Space<br/>so cookies stay first-party"]41    end42    subgraph HF["Hugging Face Space · Docker"]43        API["Flask + SSE"]44        RAG["Retrieval + generation pipeline"]45        VS[("Chroma<br/>7,694 chunks · BGE 768-dim")]46        API --> RAG --> VS47    end48    subgraph Ext["External services"]49        GEM["Gemini API<br/>selector · generator"]50        SUP[("Supabase<br/>auth · visitor history · metrics")]51    end52    UI --> CDN --> API53    RAG --> GEM54    API --> SUP55```56 57## 3. How It Works58 59Every branch below exists because a specific class of question failed without it.60 61```mermaid62flowchart TB63    Q(["User question"]) --> G{"Safety and<br/>rate limit"}64    G -->|blocked| X(["Refusal<br/>nothing retrieved"])65    G -->|ok| ST["<b>Conversation state</b><br/>resolve pronouns against the active subject"]66    ST --> LR["<b>Local router</b> — always runs<br/>classifies the question and scopes it<br/>from the entity and document registries"]67 68    LR --> ROUTE["<b>Query route</b><br/>scope · question type · facets"]69 70    ROUTE --> F{"Evidence<br/>from where?"}71    F -->|"a registry row"| EX["<b>Deterministic extractor</b><br/>staff rows, contacts, field lookups<br/><i>answer composed in code</i>"]72    F -->|"the document corpus"| RET["<b>Hybrid retrieval</b><br/>dense + BM25 + rare-term, per facet<br/><i>detailed below</i>"]73 74    RET --> SEL["<b>Evidence selector</b> (1 call)<br/>pick the answer-bearing blocks<br/>out of ~28 candidates"]75    SEL --> GEN["<b>Generation</b> (1 call)<br/>evidence-only prompt<br/>greedy decode, fixed seed"]76 77    EX -->|"no generation call"| VAL["<b>Validation</b><br/>numbers · contract · citations"]78    GEN -->|"written by the model"| VAL79    VAL --> OUT(["SSE to the browser<br/>allowlisted fields only"])80```81 82**Inside hybrid retrieval.** Three retrievers cover each other's blind spots,83then the candidate set is narrowed without letting one document dominate.84 85```mermaid86flowchart LR87    IN(["Facet query"]) --> D["Dense<br/>BGE 768-dim<br/><i>paraphrase, concept</i>"]88    IN --> B["BM25<br/><i>names, acronyms, titles</i>"]89    IN --> RP["Rare-term passage<br/><i>one sentence buried<br/>in 600 words</i>"]90    D --> FU["RRF<br/>fusion"]91    B --> FU92    RP --> FU93    FU --> RR["Rerank<br/><i>metadata, freshness,<br/>route boost</i>"]94    RR --> DD["Dedupe<br/><i>only if candidate is a<br/>subset of what is kept</i>"]95    DD --> SD["Diverse seeds<br/><i>per-document cap</i>"]96    SD --> NB["Neighbour expansion<br/><i>adjacent chunks</i>"]97    NB --> OUT(["~28 candidates"])98```99 100**The validation gauntlet.** Every draft passes five checks before it ships.101Each one exists because a specific wrong answer got through without it.102 103| # | Check | The failure it caught |104| --- | --- | --- |105| 1 | Numbers appear in the evidence | A figure in the answer that appears nowhere in the selected evidence is dropped. Percentages are checked *as percentages* — a bare `27` on a page number used to satisfy a `27%` claim |106| 2 | Answer contract | A two-part question answered in one part. "Who is X **and** what did she say?" returned only the identity |107| 3 | Drop false negatives | "The documents do not state this" while the evidence plainly stated it |108| 4 | Chunk-boundary repair | A quotation split from its attribution across two chunks, so the model declined to attribute it |109| 5 | Citations match shown sources | Markers pointing at evidence the user was never shown |110 111### What each step does, and why it is there112 113| Step | What it does | Why |114| --- | --- | --- |115| Safety + rate limit | Screens the question before anything is retrieved | Blocks abuse without spending retrieval or tokens on it |116| Conversation state | Resolves pronouns and follow-ups against the active subject | "What did she study?" would otherwise retrieve on the pronoun |117| Local router | Always runs. Classifies the question and scopes it from the entity and document registries | Produces a usable route without any model call |118| Deterministic extractor | Pulls field-style facts — names, titles, emails, counts — straight from evidence | These are already structured; generating them adds cost and risk |119| Dense + BM25 + rare-term | Three retrievers per facet | Each covers the others' blind spot: paraphrases, exact names, and single sentences diluted across 600 words |120| RRF fusion + rerank | Merges the three lists, then boosts on source, section and freshness | Fuses without needing a trained reranker |121| Dedupe + diverse seeds | Drops a chunk only when it adds nothing over one already kept, and caps any one document's share | Stops a 358-chunk document owning the whole context window |122| Neighbour expansion | Adds adjacent chunks from the same document unit | Facts span chunk boundaries |123| Evidence selector | Picks the answer-bearing blocks from ~28 candidates | 86% of measured failures had the right document in context and used a topically similar block instead |124| Chunk-boundary repair | Prepends a cut lead-in, finishes a cut sentence | A quotation split from its attribution reads as "the documents do not state this" |125| Grounded generation | Composes from the selected evidence only, greedy with a fixed seed | Same question and evidence gives the same answer, so a change can be told from noise |126| Validation | The five checks above | Retrieval finding the right text does not mean the answer used it correctly |127| Suggestions | Follow-up chips drawn from `verified_question_bank.json` | Curated and answerable, rather than invented live |128 129## 4. Features130 131The chatbot answers questions about SSL research projects, publications, staff, initiatives, funding, and community partnerships using only the lab's own source documents. Everything the model says is grounded in retrieved chunks — no free-form invention.132 133### User-Facing Features134 135- **Grounded answers** drawn directly from SSL source documents (annual reports, project pages, publications, staff bios).136- **Streaming responses** — text appears token by token as Gemini generates it, using Server-Sent Events.137- **Suggested questions** — starter buttons on first load plus verified follow-up chips after some answers.138- **Saved sessions sidebar** — one row per session, titled with the message that opened it. Clicking a row reopens that session and continues it; **+ New** starts a fresh one, and deleting asks first. Signed-in visitors keep their sessions across logins, capped at 200 saved messages each.139- **Content filter** — blocks profanity, hate speech, threats, and SSL/UMB-targeted harassment with a custom whitelist for legitimate academic terms (e.g. `assessment`, `massachusetts`, bird species) and a custom block list for org-specific phrases.140- **Friendly error handling** — Gemini 503/429 errors surface as "high demand, try again" instead of raw stack traces.141- **Citation-aware answers** — citations are normalized against the final answer and filtered to sources actually shown to the user.142- **Personal analytics dashboard** at `/dashboard`, open without a login and scoped to the caller's own activity: latency, tokens, cost, retrieval path, cited sources, corpus coverage, and low-confidence cases. Anonymous visitors see the current session only; signed-in visitors also see their saved chats. The aggregate staff view over every visitor's chats stays behind an admin session.143- **Optional visitor accounts** — signing in only controls whether a visitor's own history is saved; answers are identical either way.144 145### Document Ingestion146 147At first run, [`SEED_DOCUMENTS/`](SEED_DOCUMENTS/) is parsed into structured units:148- Project pages get split per project ([`split_project_sections`](Chatbot.py#L455)).149- Staff/board/affiliate pages get split per person with name detection ([`split_people_sections`](Chatbot.py#L590)).150- Slide decks get split per slide ([`split_slide_sections`](Chatbot.py#L547)).151- Everything else is chunked with `RecursiveCharacterTextSplitter`.152 153Each chunk is embedded and stored in ChromaDB with rich metadata (title, category, folder, source path, section name, chunk level). The metadata is what makes routing and reranking possible.154 155---156 157## 5. Models and Cost per Answer158 159Two model calls per answer. The work is split across two tiers so the expensive160model only does what needs it. A third LLM planning stage exists in the code but161is disabled — enabling it regressed 4 of 50 benchmark questions and fixed none.162 163| Stage | Model | Why this tier |164| --- | --- | --- |165| Evidence selector | `gemini-3.1-flash-lite` | Picks the answer-bearing blocks from ~28 candidates. A cheaper tier is enough — it chooses between texts, it does not write |166| Generation | `gemini-3.5-flash-lite` | Composes the grounded answer, greedy decode with a fixed seed |167| Judge *(offline only)* | `gemini-3.1-flash-lite` | Scores benchmark runs. Never called in production |168 169Published paid-tier rates, USD per 1M tokens170([pricing](https://ai.google.dev/gemini-api/docs/pricing), checked 2026-09-05).171Thinking tokens bill at the output rate:172 173| Model | Input | Output | Cached input |174| --- | --- | --- | --- |175| `gemini-3.5-flash-lite` | $0.30 | $2.50 | $0.03 |176| `gemini-3.1-flash-lite` | $0.25 | $1.50 | $0.025 |177| `gemini-3.5-flash` | $1.50 | $9.00 | $0.15 |178| `gemini-3.1-flash` | $0.75 | $3.75 | $0.075 |179| `gemini-3.1-pro` | $2.00 | $12.00 | $0.20 |180 181A typical answer runs roughly 9k input and 750 output tokens across the two182calls, which lands around **$0.004 per answer** — about 250 questions per183dollar. The dashboard reports the real figure per answer rather than an184estimate, computed from the token counts the API returns.185 186## 6. Security Model187 188| Concern | How it is handled |189| --- | --- |190| Prompt injection and abuse | Input screened before retrieval; profanity, threats and SSL-targeted harassment blocked, with a whitelist so legitimate academic terms (`assessment`, `massachusetts`, bird species) are not caught |191| Rate limiting | Per-IP limits on chat, login and signup |192| What the browser may receive | An **allowlist**, not a blocklist. Prompts, evidence text, retrieval traces, chunk scores and routing decisions never leave the server, so a new internal field cannot leak by being forgotten |193| One visitor reading another's history | Visitor tables are read and written with the visitor's own access token, never the service-role key, under `auth.uid() = user_id` policies. Postgres enforces it, so a backend bug cannot cross the boundary |194| Staff reading named visitors' chats | `flagged_chats` carries the question and retrieval trace but no user id or email, so quality review cannot become surveillance |195| Session cookies | `HttpOnly`, `Secure`, `SameSite=None`, signed with a server-side secret. `/api` is proxied through the frontend origin so the cookie is first-party rather than a third-party cookie browsers now block |196| Staff privilege escalation | Dashboard access requires `app_metadata.role = "staff"`, writable only with the service-role key. A visitor cannot promote themselves |197| Secrets | Read from the environment only. No key, hash or token is committed; `.env` is ignored and `ADMIN_USERS_JSON` stores pbkdf2 hashes, never plaintext |198 199### A deliberate exception for this demo200 201**The employee dashboard access control is fully implemented — and switched off202for the public demo on purpose.**203 204The staff dashboard aggregates every visitor's chats, and that aggregate view is205still gated: `/api/dashboard`, `/api/dashboard/interaction/<id>` and their pages206all require an admin session, exactly as built. `make_admin_users.py`, the207Supabase staff role, and the fail-closed auth path are all in place and208documented above.209 210What the demo does instead is expose a **separate personal dashboard** that211carries the same operational depth — latency, tokens, cost, retrieval path,212cited sources, corpus coverage, low-confidence cases — but213scoped to whoever is looking. That way a reviewer can see how the observability214works without being handed admin credentials, and without anyone's215conversations being published.216 217Turning the staff view back on for a real deployment is configuration, not code:218set `DASHBOARD_SESSION_SECRET` plus either `ADMIN_USERS_JSON` or a Supabase219staff role, and sign in at `/admin/login`.220 221## 7. Tech Stack222 223**Backend** — Python 3 · Flask (REST + SSE) · Google Gemini (`google-genai`) ·224ChromaDB · sentence-transformers (`BAAI/bge-base-en-v1.5`) · custom BM25 ·225langchain-text-splitters · pypdf · better-profanity · python-dotenv226 227**Frontend** — HTML / CSS / vanilla JS, no framework · Server-Sent Events ·228`fetch` + `ReadableStream` · client-side Markdown rendering229 230**Auth and data** — Supabase Auth and Postgres, row-level security231 232**Hosting** — Hugging Face Spaces (Docker) · Vercel (static frontend, `/api` proxy)233 234---235 236## 8. Evaluation237 238Tuning ran against targeted failure subsets, not aggregate scores, with239chunk-level tracing on every run so we could see which chunks retrieval240returned and which stage lost the answer. Larger subsets and full runs then241showed where overall performance actually stood. Every failure we fixed turned242out to be a structural bug — evidence mangled before the prompt, a dedupe that243deleted the longer chunk, a validator misreading `2020-21` as an invented244number, a router matching `project` inside `projected` — not a tuning gap.245 246Two 208-question sets, scored by a separate Gemini judge pass on correctness247against the corpus, citations, hallucination, and whether every part of the248question was answered. The scores count correctness failures; citation-only249mismatches, where the answer is right but cites a different valid source than250the question expected, are listed separately.251 252| Set | Before | Judge score | Citation-only | Verified defects |253| --- | --- | --- | --- | --- |254| `2026-07-11` — 160 single-turn + 48 multi-turn | 202/208 | **207/208** (48/48 multi-turn) | 3 | 0 |255| `2026-08-29` — 208 single-turn | 175 | **205/208** | 10 | 0 |256 257**Every remaining failure is a judge disagreement, each checked against the258source by hand:**259 260* `fs_111` — the per-group figures 88%, 87%, 86% are on page 32 of *Views that261  Matter*, under "has probably been happening".262* `n146` — Table 7.9's asset values are on page 134 of the cited report, and the263  same numbers passed in the previous run.264* `n156` — Figure 4 lists its four labels then its four value rows in the same265  order, giving Black 27%; the judge swapped Black and Latino/a.266* `n199` — the answer is right but drawn from a different valid document than267  the expected reference.268 269The first three share one cause: the judge is handed a corpus excerpt that does270not contain the passage the answer came from, so a correct answer scores 1 out271of 5. That is a limitation of the harness, not of the pipeline, which has no272known unfixed defect on either set.273 274The last one fixed was `n168`, which asked for Boston's projected annualized275flood losses and answered that the documents did not state them. They were in an276indexed chunk of the target publication all along: the router matched `project`277inside `projected`, scoped the question to the projects registry, and the answer278chunk was capped out of the pool for source diversity. Word-boundary matching279fixed it with no measured cost — 416 questions re-run, one newly passing, no280regressions.281 282Generation is greedy with a fixed seed, but the evidence selector is a separate283model call that can fall back on a 503, so a single run moves by about ±1284question. Compare full runs, not individual questions.285 286---287 288## 9. Running It289 290```bash291pip install -r requirements.txt292export GEMINI_API_KEY=your-key293python3 Chatbot.py            # http://localhost:7860294```295 296First run builds the vector store from `SEED_DOCUMENTS/`; after that it loads297the committed Chroma index.298 299Deployed as a Hugging Face Space (Docker, backend + vector store) with the300static frontend on Vercel, which proxies `/api` to the Space so session cookies301stay first-party. Environment variables, the Supabase schema, staff accounts302and the full deploy steps are in **[docs/DEPLOYMENT.md](docs/DEPLOYMENT.md)**.303