ComposableConsult/RAGwith3BModel

🔍 RAG Assistant – Retrieval-Augmented Generation with Uploads, Chat, and GPU Support

This app is a lightweight, production-ready Retrieval-Augmented Generation (RAG) chatbot built for Hugging Face Spaces. It allows you to upload .txt and .pdf documents, and ask questions grounded in the content of those documents. It uses local embedding + retrieval to enhance the accuracy and relevance of its answers.

🚀 Features

✅ Upload and index multiple documents (PDF and TXT supported) ✅ GPU acceleration when available (CPU fallback) ✅ Lightweight language model for cost-effective deployment (TinyLlama) ✅ Persistent chat interface for follow-up questions ✅ SentenceTransformer + FAISS for semantic retrieval ✅ Clean Gradio UI

🧠 How It Works

1. 1.Document Upload: Upload one or more .txt or .pdf files.
2. 2.Embedding: The content is split into 500-character chunks and embedded using all-MiniLM-L6-v2.
3. 3.Indexing: FAISS is used to index the embeddings for fast similarity search.
4. 4.Retrieval: When a user asks a question, the most relevant chunks are retrieved based on cosine similarity.
5. 5.Generation: A small LLM (TinyLlama-1.1B) uses the retrieved context to generate an answer.

🧱 Tech Stack

• —LLM: TinyLlama/TinyLlama-1.1B-Chat-v1.0 (can be swapped for any HF-compatible model)
• —Embeddings: sentence-transformers/all-MiniLM-L6-v2
• —Vector Store: FAISS (in-memory)
• —PDF Parsing: PyPDF2
• —UI: Gradio Blocks
• —Device: Auto GPU/CPU via torch.cuda.is_available() and device_map="auto" (requires accelerate)

🧩 Optional Extensions

• —🔁 Add support for long-term chat memory
• —💬 Replace Gradio chatbot with streaming output
• —🧠 Use quantized GGUF models via llama.cpp + ctransformers for faster CPU-only inference

🛠 Setup (Local Dev)

pip install -r requirements.txt
python app.py


Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference