CoolFace
Apppublic

prithvi55/talk2data

sourceHugging Facemitupdated 1y agoView on Hugging Face
1likes
App README

Multi-Modal RAG System

Overview data

This project is a modular, multi-modal Retrieval-Augmented Generation (RAG) system built with Streamlit. It enables natural language querying of business data, prompt tuning, PDF Q&A, and multi-agent reasoning. The system is designed for maintainability, local/cloud compatibility, and easy deployment (e.g., Hugging Face Spaces).

Features

  • —SQL Chat Agent: Chat with an agent that generates and executes SQL queries on a SQLite business database.
  • —Prompt Tuning Agent: Improve user prompts for better SQL generation using LLM.
  • —PDF Q&A Agent: Upload PDFs and ask questions; the agent uses semantic search and LLM to answer.
  • —Multi-Agent Q&A: Complex queries are handled by a multi-agent system using SQL, prompt tuning, and PDF tools.
  • —Session History: All chat interactions are preserved and displayed.
  • —Data Upload: Upload CSV, Excel, or PDF files for querying and analysis.
  • —Vector DB Integration: Prompts and SQL pairs are stored in ChromaDB for semantic search and reuse.
  • —Download/Save Results: Download query results as CSV or save as database views.

File Structure

  • —app.py: Main Streamlit app. All core logic, UI, and agent orchestration are here. Utility functions are defined locally for maintainability.
  • —db_utils.py: Database connection utilities (engine, SQLDatabase).
  • —llm_utils.py: LLM and prompt template setup.
  • —chroma_utils.py: ChromaDB client, collection, and embedding model setup.
  • —pdf_utils.py: PDF text extraction utility.
  • —requirements.txt: All required Python packages for local/cloud deployment.
  • —business_data.db: SQLite database file.
  • —prompts.jsonl: Stores prompt/SQL pairs for semantic search and history.

How It Works

  1. 1.Startup:
  2. 2.Launches Streamlit UI with custom styling.
  3. 3.Initializes database, LLM, ChromaDB, and embedding model.
  4. 4.Chat:
  5. 5.User enters a question.
  6. 6.The agent checks for schema/meta queries, direct table requests, follow-ups, semantic matches, and generates SQL using LLM.
  7. 7.Results are displayed and saved to history.
  8. 8.Prompt Tuning:
  9. 9.User can improve prompts for better SQL generation.
  10. 10.PDF Q&A:
  11. 11.Upload PDFs and ask questions; agent finds relevant context and answers using LLM.
  12. 12.Multi-Agent Q&A:
  13. 13.Handles complex queries using all available tools.
  14. 14.Data Upload:
  15. 15.Upload CSV/Excel/PDF files for analysis and querying.
  16. 16.Vector DB:
  17. 17.Prompts and SQL pairs are stored in ChromaDB for semantic search and reuse.

Setup & Usage

  1. 1.Install Requirements:
bash
   pip install -r requirements.txt
  1. 1.Run the App:
bash
   streamlit run app.py
  1. 1.Upload Data:
  2. 2.Use the sidebar to upload CSV, Excel, or PDF files.
  3. 3.Interact:
  4. 4.Use the chat input to ask questions about your data.
  5. 5.Use tabs for prompt tuning, PDF Q&A, and multi-agent queries.

Deployment

  • —Designed for local use and cloud platforms (e.g., Hugging Face Spaces).
  • —All file paths are relative for compatibility.
  • —Modular structure allows easy maintenance and extension.

Customization

  • —To add new agents/tools, define them locally in app.py or as utility modules.
  • —To change the database, update db_utils.py and the database file.
  • —To extend prompt logic, modify the prompt templates in llm_utils.py or directly in app.py.

Architecture

The Multi-Modal RAG System is designed with modularity and extensibility in mind. The architecture consists of the following layers and components:

  • —Streamlit UI Layer:
  • —Provides the chat interface, sidebar for file uploads, and tabbed navigation for different agent modes.
  • —Displays session history, query results, and agent responses.
  • —Core Logic Layer (`app.py`):
  • —Orchestrates all agent interactions, data flow, and UI updates.
  • —Contains local definitions for utility functions and agent logic for maintainability.
  • —Utility Modules:
  • —db_utils.py: Handles database engine and connection setup.
  • —llm_utils.py: Manages LLM initialization and prompt templates.
  • —chroma_utils.py: Sets up ChromaDB client, collections, and embedding models for semantic search.
  • —pdf_utils.py: Extracts text from uploaded PDF files.
  • —Agents and Tools:
  • —SQL Chat Agent: Generates and executes SQL queries using LLM and database schema.
  • —Prompt Tuning Agent: Improves user prompts for more accurate SQL generation.
  • —PDF Q&A Agent: Uses semantic search and LLM to answer questions about uploaded PDFs.
  • —Multi-Agent Q&A: Combines all tools for complex, context-aware reasoning.
  • —Data Flow:
  • —User submits a query or uploads a file via the UI.
  • —The appropriate agent/tool processes the input, using the database, LLM, or ChromaDB as needed.
  • —Results are displayed in the UI and saved to session history.
  • —Prompts and SQL pairs are stored in ChromaDB and prompts.jsonl for future semantic search and reuse.
  • —Extensibility:
  • —New agents/tools can be added as local functions or utility modules.
  • —The modular structure allows easy updates for deployment on cloud platforms or local environments.

Diagram (Textual):

User ──> Streamlit UI ──> app.py ──> [db_utils | llm_utils | chroma_utils | pdf_utils]
         │                  │
         │                  └─> Agents/Tools (SQL, Prompt, PDF, Multi-Agent)
         │
         └─> Results, History, Downloads, Views

License

MIT License

Author

pb59