ComposableConsult/RAGwith3BModel

1# 🔍 RAG Assistant – Retrieval-Augmented Generation with Uploads, Chat, and GPU Support2 3This app is a lightweight, production-ready Retrieval-Augmented Generation (RAG) chatbot built for Hugging Face Spaces. It allows you to upload `.txt` and `.pdf` documents, and ask questions grounded in the content of those documents. It uses local embedding + retrieval to enhance the accuracy and relevance of its answers.4 5---6 7## 🚀 Features8 9✅ Upload and index multiple documents (PDF and TXT supported)  10✅ GPU acceleration when available (CPU fallback)  11✅ Lightweight language model for cost-effective deployment (`TinyLlama`)  12✅ Persistent chat interface for follow-up questions  13✅ SentenceTransformer + FAISS for semantic retrieval  14✅ Clean Gradio UI15 16---17 18## 🧠 How It Works19 201. **Document Upload**: Upload one or more `.txt` or `.pdf` files.212. **Embedding**: The content is split into 500-character chunks and embedded using `all-MiniLM-L6-v2`.223. **Indexing**: FAISS is used to index the embeddings for fast similarity search.234. **Retrieval**: When a user asks a question, the most relevant chunks are retrieved based on cosine similarity.245. **Generation**: A small LLM (`TinyLlama-1.1B`) uses the retrieved context to generate an answer.25 26---27 28## 🧱 Tech Stack29 30- **LLM**: `TinyLlama/TinyLlama-1.1B-Chat-v1.0` (can be swapped for any HF-compatible model)31- **Embeddings**: `sentence-transformers/all-MiniLM-L6-v2`32- **Vector Store**: FAISS (in-memory)33- **PDF Parsing**: `PyPDF2`34- **UI**: Gradio Blocks35- **Device**: Auto GPU/CPU via `torch.cuda.is_available()` and `device_map="auto"` (requires `accelerate`)36 37---38 39## 🧩 Optional Extensions40 41- 🔁 Add support for long-term chat memory42- 💬 Replace Gradio chatbot with streaming output43- 🧠 Use quantized GGUF models via `llama.cpp` + `ctransformers` for faster CPU-only inference44 45---46 47## 🛠 Setup (Local Dev)48 49```bash50pip install -r requirements.txt51python app.py52 53 54Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference55