CoolFace
Apppublic

omravalccc/rag_final_optimized

sourceHugging Faceupdated 8mo agoView on Hugging Face
0likes
App README

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference

📚 RAG Final Optimized - Knowledge Assistant

A production-ready Retrieval-Augmented Generation (RAG) system built for efficient document processing, semantic search, and intelligent question answering. Optimized for Hugging Face Spaces deployment with Docker.

🎯 Project Overview

This system allows users to:

  • —Upload documents (PDF, DOCX, TXT, Images)
  • —Automatically extract text with OCR support
  • —Create semantic chunks using embedding similarity
  • —Store vectors in FAISS for fast retrieval
  • —Answer questions using Gemini LLM with retrieved context

🏗️ Architecture Choices & Justifications

1. Embedding Model Selection

ModelSizeSpeedQualityHF DeploymentOur Choice
intfloat/e5-base-v2438MB⚡ Fast🟢 Excellent✅ Easy✅ Selected
sentence-transformers/all-MiniLM-L6-v280MB⚡⚡ Fastest🟡 Good✅ Easy❌
text-embedding-ada-002 (OpenAI)API🟡 Network🟢 Excellent❌ API Key Required❌
BAAI/bge-large-en-v1.51.3GB🔴 Slow🟢 Excellent⚠️ Memory Issues❌
google/flan-t5-large3GB🔴 Very Slow🟢 Excellent❌ Load Time 5min+❌

Why E5-base-v2?

  • —Perfect balance of size (438MB) and quality
  • —Supports asymmetric retrieval with query: and passage: prefixes
  • —Loads in under 30 seconds on HF Spaces
  • —MTEB benchmark top performer in its size category
  • —No API key required - runs fully locally

2. Vector Database Selection

Vector DBSpeedMemoryPersistenceSetup ComplexityOur Choice
FAISS (CPU)⚡⚡ Fastest🟢 Low✅ File-based🟢 Simple✅ Selected
ChromaDB⚡ Fast🟡 Medium✅ SQLite🟡 Medium❌
Pinecone⚡ Fast☁️ Cloud✅ Managed🔴 API Key + Setup❌
Weaviate⚡ Fast🔴 High✅ Yes🔴 Complex Docker❌
Milvus⚡ Fast🔴 High✅ Yes🔴 Very Complex❌
PostgreSQL + pgvector🟡 Medium🟡 Medium✅ Yes🔴 DB Setup Required❌

Why FAISS?

  • —Zero external dependencies - just pip install
  • —File-based persistence - simple save/load
  • —Inner Product similarity for normalized embeddings
  • —Battle-tested by Meta AI for billion-scale search
  • —Works perfectly in Docker/HF Spaces environment

3. LLM Selection for Answer Generation

LLMResponse QualitySpeedCostHF DeploymentOur Choice
Gemini 2.5 Flash🟢 Excellent⚡⚡ Very Fast💚 Free Tier✅ API Key Only✅ Selected
Gemini 2.5 Pro🟢 Excellent⚡ Fast💛 Paid✅ API Key Only✅ Fallback
GPT-4o🟢 Excellent⚡ Fast🔴 Expensive✅ API Key Only❌
GPT-3.5 Turbo🟡 Good⚡⚡ Very Fast💛 Cheap✅ API Key Only❌
Llama 3.1 70B (Local)🟢 Excellent🔴 Very Slow💚 Free❌ 140GB+ RAM❌
Mistral 7B (Local)🟡 Good🔴 Slow💚 Free❌ 16GB+ RAM❌
google/flan-t5-large🟡 Medium🔴 Very Slow💚 Free⚠️ 5min+ Load❌

Why Gemini 2.5 Flash?

  • —Free tier with generous quota
  • —Extremely fast response times
  • —High quality outputs for RAG tasks
  • —No model download required - API based
  • —Automatic fallback to Pro and older versions
  • —Easy HF Spaces deployment with just API key in secrets

4. Chunking Strategy Comparison

StrategyContext PreservationSpeedComplexityOur Choice
Semantic Chunking🟢 Excellent🟡 Medium🟡 Medium✅ Selected
Fixed Size (500 tokens)🔴 Poor⚡⚡ Fastest🟢 Simple❌
Sentence-based🟡 Good⚡ Fast🟢 Simple❌
Paragraph-based🟡 Good⚡ Fast🟢 Simple❌
Recursive Character🟡 Good⚡ Fast🟡 Medium❌

Why Semantic Chunking?

  • —Groups semantically similar sentences together
  • —Uses embedding similarity threshold (0.75)
  • —Preserves topic coherence within chunks
  • —Better retrieval accuracy than fixed-size chunking
  • —Respects natural content boundaries

5. OCR and Document Processing

ToolPDF SupportImage OCRDOCXSpeedOur Choice
pdfplumber + pytesseract✅ Full✅ Full✅ Full🟡 Medium✅ Selected
PyMuPDF (fitz)✅ Full⚠️ Limited❌⚡ Fast❌
Unstructured.io✅ Full✅ Full✅ Full🔴 Slow❌
Azure Document Intelligence✅ Full✅ Full✅ Full⚡ Fast❌ API Cost
AWS Textract✅ Full✅ Full⚠️ Limited⚡ Fast❌ API Cost

Why pdfplumber + pytesseract?

  • —Open source and free
  • —Excellent PDF text extraction
  • —Tesseract OCR for scanned pages and images
  • —Works in Docker with apt packages
  • —Handles mixed documents (text + images)

📊 Performance Benchmarks

Model Loading Times (HF Spaces - 2 vCPU, 16GB RAM)

ComponentLoad TimeMemory
E5-base-v2 Embedding~25 seconds~900MB
FAISS Index (10K chunks)< 1 second~50MB
Streamlit App~5 seconds~200MB
Total Cold Start~35 seconds~1.2GB

Query Performance

OperationTime
Embed Query~50ms
FAISS Search (10K vectors)~5ms
Gemini API Response~1-3 seconds
Total Query Time~2-4 seconds

🔄 Why NOT These Alternatives?

Local LLMs (Llama, Mistral, etc.)

IssueImpact
Model Size7B+ models = 14GB+ download
RAM Requirements16-32GB minimum
HF Spaces LimitsFree tier has 16GB RAM
Cold Start5-10 minutes to load
Inference Speed10-60 seconds per response

OpenAI Embeddings

IssueImpact
API Costs$0.0001 per 1K tokens
Network DependencyLatency + potential failures
Rate LimitsCan hit limits on batch processing
Data PrivacyData sent to OpenAI servers

Google Flan-T5 Models

IssueImpact
Load Time3-8 minutes on HF Spaces
Memory Usage3-6GB for large variants
Response QualityGood but not as detailed as Gemini
HF DeploymentOften times out during loading

Heavy Vector DBs (Milvus, Weaviate)

IssueImpact
Setup ComplexityRequires separate containers
Memory Usage2-4GB minimum
Docker ComposeNot supported on HF Spaces
OverkillOur scale doesn't need distributed DB

🛠️ Technical Implementation Details

Embedding Strategy

Documents → "passage: {text}" → E5-base-v2 → Normalized Vectors
Queries   → "query: {text}"   → E5-base-v2 → Normalized Vectors

Similarity Metric

  • —Using Inner Product (IP) instead of L2 distance
  • —Vectors are normalized → IP equals cosine similarity
  • —Faster computation for normalized vectors

Chunking Algorithm

1. Split text into sentences
2. Embed each sentence
3. For each sentence:
   - If similarity with current chunk ≥ 0.75 AND size < max_size:
     → Add to current chunk
   - Else:
     → Start new chunk
4. Merge small chunks (< 100 chars) with neighbors

📁 Project Structure

rag_final_optimized/
├── utils/
│   ├── docs_reciever.py     # Document ingestion + chunking + embedding
│   └── ans_retrieval.py     # Vector search + Gemini answer generation
├── streamlit.py             # Web UI with tabs
├── requirements.txt         # Python dependencies
├── packages.txt             # System packages (apt)
├── Dockerfile               # HF Spaces deployment
├── .env                     # API keys (local only)
└── README.md                # This file

🚀 Deployment

Hugging Face Spaces

  1. 1.Push to HF Space repository
  2. 2.Add GEMINI_API_KEY to Space secrets
  3. 3.Wait for Docker build (~5 minutes)
  4. 4.App is live!

Local Development

bash
pip install -r requirements.txt
streamlit run streamlit.py

🔑 Environment Variables

VariableRequiredDescription
GEMINI_API_KEY✅ YesPrimary Gemini API key
GEMINI_API_KEY_2❌ OptionalFallback API key for rate limits

📈 Future Improvements

  • —[ ] Add reranking with cross-encoder
  • —[ ] Implement hybrid search (BM25 + vector)
  • —[ ] Add conversation history/memory
  • —[ ] Support more file formats (Excel, PPT)
  • —[ ] Add multi-language support

📝 License

MIT License - Feel free to use and modify!

Built with ❤️ using Streamlit, FAISS, E5 Embeddings, and Gemini