bhanu-marisa/simple-rag

1---2title: Simple RAG3emoji: 🔍4colorFrom: blue5colorTo: purple6sdk: streamlit7sdk_version: 1.45.08app_file: app.py9pinned: false10---11 12# Simple RAG13 14A learning project to understand and build a Retrieval Augmented Generation (RAG) pipeline from scratch, progressively adding complexity.15 16## What is RAG?17 18LLMs like GPT-4 are trained on general data and have a knowledge cutoff. RAG solves two problems:19- Answering questions from **your own documents** the LLM has never seen20- Grounding answers in a specific source so the LLM doesn't hallucinate21 22Instead of sending all your documents to the LLM every time (expensive, hits token limits), RAG retrieves only the relevant chunks and sends those.23 24```25Your documents → split into chunks → embed as vectors → store in ChromaDB26                                                               ↓27User query → embed → similarity search → relevant chunks → LLM → answer28```29 30## How this app works31 321. **Documents** — `.txt` files in `docs/` are the knowledge base332. **Embeddings** — `HuggingFaceEmbeddings` (all-mpnet-base-v2) converts text into 768-dimensional vectors locally, no API needed343. **ChromaDB** — stores the vectors on disk, loaded on restart without re-embedding354. **Retrieval** — hybrid search combining BM25 (keyword) + MMR (semantic vector search)365. **Re-ranking** — FlashrankRerank re-scores retrieved chunks for better relevance376. **Agentic loop** — LangGraph agent grades chunks, rewrites query and retries if needed387. **Generation** — relevant chunks passed to `gpt-4o-mini`, answers only from context with source citations39 40## Project structure41 42```43simple-rag/44├── agent.py                      # RAG logic — LangGraph agentic pipeline45├── api.py                        # FastAPI — exposes RAG as HTTP endpoints46├── ui.py                         # Streamlit — chat UI, calls FastAPI47├── scraper.py                    # downloads and cleans Wikipedia docs48├── docs/                         # knowledge base (txt files)49│   ├── transformer.txt           # Wikipedia: Transformer architecture50│   ├── large_language_model.txt  # Wikipedia: Large language models51│   └── cognitive_psychology.txt  # Wikipedia: Cognitive psychology52├── chroma_db/                    # auto-generated, persisted vector store53├── requirements.txt54├── .env                          # API keys, never commit55└── .gitignore56```57 58## Setup59 60```bash61git clone <repo>62cd simple-rag63 64python3 -m venv venv65source venv/bin/activate66 67pip install -r requirements.txt68```69 70Create a `.env` file:71```72OPENAI_API_KEY=sk-...73```74 75Download and clean docs:76```bash77python3 scraper.py78```79 80Run — requires two terminals:81```bash82# Terminal 1 — FastAPI backend83uvicorn api:app --reload84 85# Terminal 2 — Streamlit UI86streamlit run ui.py87```88 89First run builds the vector store (~30 seconds). Every run after loads instantly.90 91Streamlit opens at `http://localhost:8501`. FastAPI docs at `http://localhost:8000/docs`.92 93To add new documents, drop `.txt` files into `docs/`, delete `chroma_db/`, and rerun.94 95## Key concepts96 97| Concept | What it does in this app |98|---|---|99| Embeddings | Converts text to numbers that capture meaning |100| Cosine similarity | Measures how similar two vectors are by angle, not size |101| Chunking | Splits large docs into pieces so retrieval is precise |102| Chunk overlap | Prevents meaning loss at chunk boundaries |103| Persistent ChromaDB | Avoids re-embedding on every restart |104| Score threshold | Filters out irrelevant chunks before sending to LLM |105| Prompt template | Instructs LLM to answer only from provided context |106 107## Tuning knobs108 109- `chunk_size` — larger chunks = more context per chunk, but less precise retrieval110- `chunk_overlap` — higher overlap = less meaning lost at boundaries, but more redundancy111- `k` in `similarity_search_with_score(k=3)` — how many chunks to retrieve112- score threshold `1.2` — lower = stricter (fewer results), higher = looser (more results)113 114## Roadmap115 116**Level 2 — Real documents**117- [x] Load from txt files118- [x] Text chunking with overlap119- [x] Persistent ChromaDB120 121**Level 3 — Better retrieval**122- [x] Metadata filtering by filename/topic123- [x] MMR (Maximal Marginal Relevance) — avoid redundant chunks124- [x] Hybrid search — keyword + vector combined125 126**Level 4 — Better generation**127- [x] Conversation memory across questions128- [x] Source citations in answers129- [x] Streaming responses130 131**Current: Level 5 — Production patterns**132- [x] Query rewriting133- [x] Re-ranking with a second model134- [x] Agentic RAG135