CoolFace
Apppublic

ShawnIL/GAIA_Agent_Evaluation

sourceHugging Faceupdated 8mo agoView on Hugging Face
0likes
App README

yy

🤖 GAIA Agent - Advanced Q&A Chatbot

A sophisticated AI Agent powered by LangGraph, LangChain, and Groq LLM with RAG capabilities, 21+ integrated tools, and comprehensive observability through Langfuse.

🌟 Overview

GAIA Agent is a production-ready AI chatbot that combines:

  • —🧠 Advanced Reasoning: Groq qwen3-32b LLM with tool-calling
  • —📚 RAG (Retrieval-Augmented Generation): Supabase vector DB for contextual knowledge
  • —🛠️ 21+ Tools: Web search, code execution, file processing, image analysis
  • —🎨 Modern UI: Gradio 6.0 interface with file upload support
  • —📊 Observability: Langfuse integration for LLM monitoring
  • —⚡ Fast: RPC-based Supabase retrieval for speed

🚀 Key Features

Core Capabilities

  • —🔍 Multi-Source Search: Tavily web search, Wikipedia, arXiv papers
  • —💻 Code Execution: Python, Bash, SQL, C, Java with sandboxed execution
  • —🖼️ Image Processing: OCR, analysis, transformations, generation
  • —📄 Document Intelligence: PDF, CSV, Excel analysis with pandas
  • —🧮 Mathematical Tools: Complete math operations suite
  • —📁 File Upload: Drag-and-drop support for multiple file types

Technical Stack

  • —LLM: Groq (qwen3-32b) for fast inference
  • —Orchestration: LangGraph StateGraph for agent workflow
  • —Vector DB: Supabase with custom RPC for similarity search
  • —Embeddings: HuggingFace sentence-transformers (768-dim)
  • —UI: Gradio 6.0 with custom CSS and message format
  • —Monitoring: Langfuse for LLM tracing and observability

🏗️ Architecture

┌─────────────────────────────────────────────────────────────┐
│                    GAIA Agent Flow                          │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  User Question                                              │
│       │                                                     │
│       ▼                                                     │
│  ┌─────────────┐                                            │
│  │  Retriever  │  ← Supabase RPC (match_documents_2)       │
│  │    Node     │    Semantic search for similar Q&A        │
│  └─────────────┘                                            │
│       │                                                     │
│       ▼                                                     │
│  ┌─────────────┐                                            │
│  │  Assistant  │  ← Groq qwen3-32b + 21 tools bound        │
│  │    Node     │    Langfuse callback for tracing          │
│  └─────────────┘                                            │
│       │                                                     │
│       ├─── Tool Calls? ──┐                                  │
│       │                  │                                  │
│       ▼                  ▼                                  │
│   Response          ┌─────────────┐                         │
│                     │    Tools    │                         │
│                     │    Node     │                         │
│                     └─────────────┘                         │
│                          │                                  │
│                          └─── Back to Assistant             │
│                                                             │
└─────────────────────────────────────────────────────────────┘

📦 Project Structure

AI_Agents_Tutorial/
├── app.py                    # Main Gradio Q&A interface (Gradio 6.0)
├── evaluation_app.py         # GAIA benchmark evaluation runner
├── agent.py                  # Core agent: LangGraph + tools + RAG
├── code_interpreter.py       # Multi-language code execution
├── image_processing.py       # Image processing utilities
├── system_prompt.txt         # System instructions for LLM
├── requirements.txt          # Python dependencies
├── .env                      # Environment variables (NOT in git)
├── metadata.jsonl           # GAIA benchmark metadata
├── ARCHITECTURE.md          # System architecture documentation
├── DATA_FLOW.md            # Data flow diagrams
├── DEVELOPMENT_GUIDE.md    # Comprehensive development guide
└── README.md               # This file

🛠️ Complete Tool Inventory (21 Tools)

🌐 Search & Research (3 tools)

  1. 1.web_search: Tavily-powered real-time web search
  2. 2.wiki_search: Wikipedia article retrieval
  3. 3.arxiv_search: Academic paper search

💻 Code Execution (5 tools)

  1. 1.execute_code_multilang: Python, Bash, SQL, C, Java execution
  2. 2.add: Addition operation
  3. 3.subtract: Subtraction operation
  4. 4.multiply: Multiplication operation
  5. 5.divide: Division with zero-check

📄 File & Data Processing (6 tools)

  1. 1.save_file: Write content to file
  2. 2.read_file: Read file content
  3. 3.download_file: Download from URL
  4. 4.analyze_csv: Pandas-based CSV analysis
  5. 5.analyze_excel: Excel file processing
  6. 6.extract_text_from_pdf: PDF text extraction

🖼️ Image Processing (7 tools)

  1. 1.analyze_image: Size, format, color analysis
  2. 2.resize_image: Image resizing
  3. 3.rotate_image: Rotation transformation
  4. 4.crop_image: Image cropping
  5. 5.flip_image: Horizontal/vertical flip
  6. 6.adjust_brightness: Brightness control
  7. 7.extract_text_from_image: OCR with Tesseract

🎯 Usage Examples

Example 1: Simple Question

User: "What is the capital of France?"
Agent: Uses RAG retriever → Finds similar Q&A → Returns "Paris"

Example 2: Web Search

User: "What's the current Bitcoin price?"
Agent: Detects need for real-time data → Calls web_search tool → Returns latest price

Example 3: Code Execution

User: "Calculate fibonacci sequence up to 10"
Agent: Generates Python code → Executes via execute_code_multilang → Returns [0,1,1,2,3,5,8,13,21,34]

Example 4: File Analysis

User: [Uploads sales.csv] "What are the top 3 products?"
Agent: Calls analyze_csv → Pandas analysis → Returns top products with sales figures

Example 5: Image OCR

User: [Uploads screenshot] "Extract the text from this image"
Agent: Calls extract_text_from_image → Tesseract OCR → Returns extracted text

🔧 Technical Details

LangGraph Implementation

python
# Graph structure (agent.py)
builder = StateGraph(MessagesState)
builder.add_node("retriever", retriever_node)   # RAG search
builder.add_node("assistant", assistant_node)   # LLM + tools
builder.add_node("tools", ToolNode(tools))      # Tool execution

builder.add_edge(START, "retriever")
builder.add_edge("retriever", "assistant")
builder.add_conditional_edges("assistant", tools_condition)
builder.add_edge("tools", "assistant")

RAG with Supabase RPC

python
# Custom RPC-based retriever (avoids supabase-py version issues)
def supabase_similarity_search(query: str, k: int = 3):
    embedding = embeddings.embed_query(query)
    result = supabase.rpc("match_documents_2", {
        "query_embedding": embedding,
        "match_count": k
    }).execute()
    return result.data

Langfuse Observability

python
# Optional callback for LLM tracing
lf_handler = get_langfuse_handler()  # Reads from .env
llm = ChatGroq(model="qwen/qwen3-32b", callbacks=[lf_handler])
result = graph.invoke({"messages": messages}, config={"callbacks": [lf_handler]})

🚨 Troubleshooting

Common Issues

1. Port already in use

bash
# Kill process on port 7860
lsof -ti:7860 | xargs kill -9

2. Supabase connection error

bash
# Check environment variables
echo $SUPABASE_URL
echo $SUPABASE_SERVICE_ROLE_KEY

# Test connection
python -c "from supabase import create_client; import os; client = create_client(os.getenv('SUPABASE_URL'), os.getenv('SUPABASE_SERVICE_ROLE_KEY')); print('Connected!')"

3. Gradio version mismatch

bash
# Ensure Gradio 4.x (not 6.x for compatibility)
pip install "gradio>=4.0.0,<5.0.0"

4. Missing dependencies

bash
# Install all requirements
pip install -r requirements.txt

# Install system dependencies (macOS)
brew install tesseract  # For OCR

📊 Performance Metrics

MetricValue
LLM Latency~1-2s (Groq inference)
RAG Search~200ms (Supabase RPC)
Tool ExecutionVariable (depends on tool)
Total Response2-5s average
Tools Available21
Supported Languages5 (Python, Bash, SQL, C, Java)
File Types15+ formats

🔗 Resources & Documentation

  • —[ARCHITECTURE.md](ARCHITECTURE.md): System architecture deep-dive
  • —[DATA_FLOW.md](DATA_FLOW.md): Data flow diagrams and explanations
  • —[DEVELOPMENT_GUIDE.md](DEVELOPMENT_GUIDE.md): Complete development guide
  • —[LangGraph Docs](https://langchain-ai.github.io/langgraph/): Official documentation
  • —[Groq](https://groq.com/): Fast LLM inference
  • —[Supabase](https://supabase.com/): Vector database
  • —[Langfuse](https://langfuse.com/): LLM observability

🤝 Contributing

Contributions are welcome! Areas for improvement:

  • —✨ New specialized tools
  • —🎨 UI/UX enhancements
  • —⚡ Performance optimizations
  • —📚 Documentation expansion
  • —🧪 Test coverage

📄 License

MIT License - See LICENSE file for details.

🙏 Acknowledgments

Built with:

  • —LangChain & LangGraph for agent orchestration
  • —Groq for blazing-fast LLM inference
  • —Supabase for vector database
  • —Gradio for beautiful UI
  • —Hugging Face for embeddings and hosting

Last Updated: January 2026 Version: 1.0.0

⚙️ Installation & Setup

Prerequisites

  • —Python 3.11+
  • —Conda (recommended) or venv
  • —API Keys: Groq, Tavily, Supabase, Langfuse (optional)

1. Clone Repository

bash
git clone <your-repo-url>
cd AI_Agents_Tutorial

2. Create Conda Environment

bash
# Create environment
conda create -p ./.conda python=3.11 -y
conda activate ./.conda

# Install dependencies
pip install -r requirements.txt

3. Environment Variables

Create .env file in project root:

env
# LLM Provider
GROQ_API_KEY=gsk_your_groq_api_key

# Search API
TAVILY_API_KEY=tvly-your_tavily_key

# Vector Database
SUPABASE_URL=https://your-project.supabase.co
SUPABASE_SERVICE_ROLE_KEY=eyJhbG...your_key

# Observability (Optional)
LANGFUSE_HOST=https://cloud.langfuse.com
LANGFUSE_PUBLIC_KEY=pk-lf-...
LANGFUSE_SECRET_KEY=sk-lf-...

4. Supabase Database Setup

Execute this SQL in your Supabase SQL Editor:

sql
-- Enable pgvector extension
CREATE EXTENSION IF NOT EXISTS vector;

-- Create documents table (if not exists)
CREATE TABLE IF NOT EXISTS public.documents2 (
  id BIGSERIAL PRIMARY KEY,
  content TEXT,
  metadata JSONB,
  embedding VECTOR(768)
);

-- Create RPC function for similarity search
CREATE OR REPLACE FUNCTION public.match_documents_2(
  query_embedding VECTOR(768),
  match_count INT DEFAULT 3
)
RETURNS TABLE(
  id BIGINT,
  content TEXT,
  metadata JSONB,
  similarity FLOAT
)
LANGUAGE plpgsql
AS $$
BEGIN
  RETURN QUERY
  SELECT
    documents2.id,
    documents2.content,
    documents2.metadata,
    1 - (documents2.embedding <=> query_embedding) AS similarity
  FROM public.documents2
  ORDER BY documents2.embedding <=> query_embedding
  LIMIT match_count;
END;
$$;

-- Create index for faster searches
CREATE INDEX IF NOT EXISTS documents2_embedding_idx 
ON public.documents2 
USING ivfflat (embedding vector_cosine_ops)
WITH (lists = 100);

-- Grant permissions
GRANT EXECUTE ON FUNCTION public.match_documents_2(VECTOR, INT) 
TO anon, authenticated, service_role;

🚀 Running the Application

Main Q&A Chatbot Interface

bash
# Activate environment
conda activate ./.conda

# Run the app
python app.py

Access at: http://localhost:7860

Features:

  • —💬 Natural language Q&A
  • —📁 Drag-and-drop file upload (images, CSV, PDF, etc.)
  • —🔍 Real-time web search
  • —💻 Code execution with results
  • —📊 Data analysis and visualization
  • —🧹 Clear history and export chat

Evaluation Runner (GAIA Benchmark)

bash
python evaluation_app.py

Workflow:

  1. 1.Login with Hugging Face account
  2. 2.Click "Run Evaluation & Submit All Answers"
  3. 3.Agent processes all GAIA benchmark questions
  4. 4.Automatic submission and scoring

🎯 Usage Examples

Example 1: Simple Question

User: "What is the capital of France?"
Agent: Uses RAG retriever → Finds similar Q&A → Returns "Paris"

Example 2: Web Search

User: "What's the current Bitcoin price?"
Agent: Detects need for real-time data → Calls web_search tool → Returns latest price

Example 3: Code Execution

User: "Calculate fibonacci sequence up to 10"
Agent: Generates Python code → Executes via execute_code_multilang → Returns [0,1,1,2,3,5,8,13,21,34]

Example 4: File Analysis

User: [Uploads sales.csv] "What are the top 3 products?"
Agent: Calls analyze_csv → Pandas analysis → Returns top products with sales figures

Example 5: Image OCR

User: [Uploads screenshot] "Extract the text from this image"
Agent: Calls extract_text_from_image → Tesseract OCR → Returns extracted text

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference