aimprabu/RAG_Documentation_Assistant

RAG Assistant: Technical Documentation Copilot

A self-correcting Retrieval-Augmented Generation (RAG) assistant built using FastAPI, LangGraph, ChromaDB, and the Gemini / Groq LLM API.

This project implements an intelligent technical assistant that can reliably answer questions over a set of ingested markdown documentation. It features an advanced self-evaluating workflow that checks document relevance, prevents hallucinations, performs web search fallback, and gracefully falls back when no relevant information is found.

Architecture

LangGraph Workflow

The core of this system is a self-corrective agentic loop built on LangGraph's StateGraph. Each node performs a specific reasoning step, with conditional edges enabling automatic retry and self-healing:

graph TD
    Q["❓ User Question"] --> QA["🔍 Query Analysis"]
    QA --> R["📚 Retrieval<br/><i>ChromaDB + Embeddings</i>"]
    R --> G["⚖️ Document Grading<br/><i>LLM Relevance Check</i>"]
    G -->|"Relevant docs found"| GEN["✨ Generation<br/><i>LLM Answer Synthesis</i>"]
    G -->|"No relevant docs<br/>retries remaining"| RW["🔄 Query Rewrite<br/><i>LLM Reformulation</i>"]
    RW -->|"Retry loop"| R
    G -->|"Max retries reached"| WS["🌐 Web Search<br/><i>DuckDuckGo / Tavily</i>"]
    WS -->|"Results found"| GEN
    WS -->|"No results"| FB["⚠️ Fallback Response"]
    GEN --> HC["🛡️ Hallucination Check<br/><i>Grounding Verification</i>"]
    HC -->|"✅ Grounded"| A["✅ Final Answer"]
    HC -->|"❌ Not Grounded"| GEN
    FB --> A

    style Q fill:#4A90D9,color:#fff
    style A fill:#27AE60,color:#fff
    style FB fill:#E67E22,color:#fff
    style WS fill:#3498DB,color:#fff
    style RW fill:#E74C3C,color:#fff
    style HC fill:#8E44AD,color:#fff

System Components

Key Design Decisions

1. 1.Local Embeddings: Uses all-MiniLM-L6-v2 locally instead of API-based embeddings. Zero cost, zero latency variance, works offline.
2. 2.Lazy LLM Initialization: LLM client is initialized once at startup and shared across requests via dependency injection singletons.
3. 3.Self-Corrective Loop: When document grading finds all chunks irrelevant, the query is automatically rewritten and re-retrieved up to MAX_RETRIES times before falling back to web search.
4. 4.Web Search Fallback: After exhausting retries, the system queries DuckDuckGo (or Tavily) and converts web results into document chunks for answer synthesis.
5. 5.Hallucination Guard: Post-generation grounding verification ensures the answer is strictly supported by retrieved context.
6. 6.Duplicate Detection: Document ingestion uses SHA-256 content hashing to prevent re-indexing identical files.

Features

• —Document Ingestion Pipeline:
• —Loads local markdown, PDF, and text documents
• —Generates embeddings locally using sentence-transformers/all-MiniLM-L6-v2
• —Persists vector data to ChromaDB with metadata
• —Maintains document registry in SQLite with duplicate detection
• —Agentic Workflow (LangGraph):
• —Query Analysis: Interprets and classifies user intent
• —Retrieval: Fetches relevant chunks from ChromaDB
• —Document Grading: LLM evaluates chunk relevance, triggers query rewrite if needed
• —Generation: Synthesizes a factual, grounded answer
• —Hallucination Check: Verifies the answer is strictly supported by retrieved context
• —Web Search Fallback: When all retries are exhausted, searches the web via DuckDuckGo/Tavily
• —Conversation Memory:
• —Session-based chat history stored in SQLite
• —Follow-up questions inherit conversation context
• —Interactive Streamlit UI:
• —Document upload and corpus browser
• —Chat interface with streaming responses
• —Retrieval Debug Panel showing full LangGraph execution trace
• —Inline feedback submission
• —REST API:
• —POST /query — Ask questions with full debug trace
• —POST /ingest — Upload and index documents
• —GET /documents — Browse indexed corpus
• —POST /feedback — Submit answer ratings
• —GET /metrics — System statistics dashboard
• —GET /health — Service health check
• —Production-Ready:
• —Automated CI/CD pipeline via GitHub Actions
• —Containerized deployment with Docker and Docker Compose
• —\>80% test coverage using Pytest

Getting Started

Prerequisites

• —Python 3.11+
• —Docker (Optional for container deployment)
• —A valid Google Gemini API Key

Installation

1. 1.Clone and setup environment:

   git clone https://github.com/lokeshtheprogrammer/Langgraph-RAG-Docs-Assistant.git
   cd Langgraph-RAG-Docs-Assistant
   python -m venv venv
   source venv/bin/activate  # Or `venv\Scripts\activate` on Windows
   pip install -r requirements.txt

1. 1.Configure Environment: Create a .env file in the root directory:

   LLM_PROVIDER=google
   LLM_MODEL=gemini-2.5-flash
   GEMINI_API_KEY=your_api_key_here
   CHROMA_PERSIST_DIR=./chroma_db
    SQLITE_DB_PATH=./data/app.db

    # Optional: Web search fallback
    WEB_SEARCH_ENABLED=true
    WEB_SEARCH_PROVIDER=duckduckgo  # or "tavily"
    TAVILY_API_KEY=your_tavily_key_here

1. 1.Ingest Documents:

   python -m ingestion.ingest_corpus

1. 1.Run the API Server:

   uvicorn app.main:app --reload --host 127.0.0.1 --port 8000

1. 1.Launch the Streamlit UI:

   streamlit run streamlit_app.py

Open http://localhost:8501 in your browser.

Docker Deployment

docker-compose up --build

• —API: http://localhost:8000
• —Swagger Docs: http://localhost:8000/docs

Testing

# Run full test suite with coverage
python -m pytest --cov=app tests/

# Smoke tests against running server
python scripts/smoke_test.py

Project Structure

RAG/
├── app/
│   ├── api/
│   │   ├── routes/          # FastAPI endpoint handlers
│   │   └── schemas/         # Pydantic request/response models
│   ├── core/                # Logging, database, exceptions, middleware
│   ├── infrastructure/      # Adapters: LLM, embeddings, vector store, doc loaders, web search
│   ├── repositories/        # SQLite data access layer
│   ├── services/            # Business logic orchestration
│   ├── utils/               # Chunking, hashing utilities
│   └── workflow/            # LangGraph nodes, routing, state, prompts
├── corpus/                  # Source documents for ingestion
├── tests/                   # Unit, integration, and API tests
├── scripts/                 # Operational utilities
├── streamlit_app.py         # Interactive Streamlit UI
├── Dockerfile               # Container build
├── docker-compose.yml       # Container orchestration
└── requirements.txt         # Python dependencies

Architecture Decision Records (ADRs)

ADR-001: LangGraph StateGraph over LangChain LCEL

Context: LangChain's LCEL (LangChain Expression Language) is the standard approach for building LLM chains, but our RAG pipeline requires cycles — when document grading fails, the query must be rewritten and retrieval re-executed, potentially multiple times.

Decision: Use LangGraph's StateGraph with conditional edges and cycle support.

Rationale:

• —LCEL pipelines are DAGs (directed acyclic graphs) — they cannot express retry loops or conditional backtracking
• —LangGraph supports cycles natively via add_conditional_edges, enabling the grading → rewrite → retrieval retry loop
• —The TypedDict state schema makes it easy to track retry_count, should_fallback, and intermediate results across nodes
• —LangGraph's ainvoke() provides async execution with full state traceability

Tradeoff: LangGraph has a steeper learning curve and fewer community examples compared to LCEL chains.

ADR-002: Local Embeddings (all-MiniLM-L6-v2) over API-Based Embeddings

Context: Embedding models can be run locally (sentence-transformers) or via API (OpenAI text-embedding-3-small, Cohere).

Decision: Use sentence-transformers/all-MiniLM-L6-v2 running locally.

Rationale:

• —Zero cost: No per-token embedding fees; critical for a submission project with unlimited test queries
• —Zero latency variance: Local inference is ~5ms per query vs. 100-300ms for API round-trips
• —Offline capability: Works without internet, simplifying reviewer setup
• —Deterministic: Same input always produces the same embedding, making tests reproducible
• —384-dimensional output: Compact vectors that are efficient for ChromaDB HNSW indexing

Tradeoff: Lower semantic quality than OpenAI's 1536-dim embeddings for nuanced queries, but sufficient for technical documentation retrieval.

ADR-003: ChromaDB over Pinecone / Weaviate / Qdrant

Context: Multiple vector databases exist with varying deployment models (managed cloud vs. self-hosted).

Decision: Use ChromaDB with local persistent storage.

Rationale:

• —Zero infrastructure: No API keys, no cloud accounts, no Docker required — just pip install chromadb
• —Persistent storage: Survives process restarts via PersistentClient backed by SQLite + Parquet
• —Native Python: First-class Python API with no REST/gRPC overhead for local usage
• —HNSW indexing: Uses hnswlib for approximate nearest neighbor search with configurable ef and M parameters
• —Reviewer-friendly: Clone, install, run — no external service dependencies

Tradeoff: Not suitable for production workloads exceeding ~1M vectors or requiring distributed search. For this assignment's corpus size (<1000 chunks), ChromaDB is ideal.

ADR-004: SQLite over PostgreSQL

Context: The system needs persistent storage for document metadata, user feedback, conversation history, and query logs.

Decision: Use SQLite with sqlite3 standard library module.

Rationale:

• —Zero dependency: Ships with Python; no database server installation required
• —Single-file database: The entire state is in data/app.db, easily backed up or reset
• —Thread-safe: Using sqlite3.connect() per-request with context managers avoids connection pooling complexity
• —Schema evolution: Simple DDL scripts executed idempotently at startup
• —Assignment scope: The workload is single-user, low-concurrency — SQLite handles this effortlessly

Tradeoff: SQLite does not support concurrent writes efficiently. A production system serving multiple users would migrate to PostgreSQL with connection pooling (e.g., asyncpg + SQLAlchemy).

ADR-005: Recursive Character Splitting over Fixed-Size Chunking

Context: Document chunking strategy directly impacts retrieval quality. Chunks that are too large dilute relevance; chunks that are too small lose context.

Decision: Use LangChain's RecursiveCharacterTextSplitter with markdown-aware separators.

Rationale:

• —Structural awareness: Splits on \n\n (paragraphs) → \n (lines) → ` (code blocks) → . (sentences) → (words), preserving logical boundaries
• —Configurable overlap: 64-character overlap ensures context continuity across chunk boundaries
• —512-char default size: Balances between retrieval precision (smaller) and context richness (larger); tunable via CHUNK_SIZE environment variable
• —Markdown-optimized: The separator hierarchy naturally respects heading sections and code blocks common in technical documentation

Tradeoff: Recursive splitting can occasionally break mid-sentence at the word-level fallback. Semantic chunking (splitting by topic similarity) would produce higher-quality chunks but adds model inference overhead during ingestion.

Documentation

License

MIT