CoolFace
Apppublic

Faraz618/VisualRAG

sourceHugging Facemitupdated 4mo agoView on Hugging Face
0likes
App README

๐Ÿ” VisualRAG โ€” Multi-Modal AI System

A production-grade Retrieval-Augmented Generation (RAG) system combining computer vision and natural language understanding.

๐Ÿง  Pipeline

Index: Image โ†’ YOLOv8 detection โ†’ CLIP ViT-B/32 embedding โ†’ FAISS vector store Query: Text โ†’ CLIP text embedding โ†’ cosine k-NN โ†’ Zephyr-7B answer generation

๐Ÿ›  Stack

ComponentTechnology
Object detectionYOLOv8n (Ultralytics)
Visual embeddingCLIP ViT-B/32 (OpenAI)
Vector indexFAISS IndexFlatIP
LLMZephyr-7B-ฮฒ (HF Serverless API)
UIGradio 4.40.0

๐Ÿš€ How to use

  1. 1.Detect & Index โ€” upload images; YOLOv8 detects objects, CLIP stores 512-d embeddings in FAISS
  2. 2.Query (RAG) โ€” ask a question; CLIP retrieves relevant images, Zephyr-7B answers
  3. 3.How it works โ€” full architecture overview

๐Ÿ”‘ Optional: HF token

Settings โ†’ Variables and secrets โ†’ New secret โ†’ Name: HF_TOKEN