aswyga/agent_bench

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference

AI Benchmark Agent

A streamlined, multi-tool AI agent optimized for passing AI benchmarks like GAIA. Built with LangChain and LangGraph, featuring essential tools for information retrieval, code execution, and mathematical operations.

🚀 Features

• —Multi-language Code Execution: Python, Bash, SQL, C, Java
• —Information Retrieval: Web search and Wikipedia lookup
• —Mathematical Operations: Basic arithmetic, powers, roots, modulus
• —File Processing: CSV/Excel analysis, file operations
• —Optimized for Benchmarks: Focused toolset for maximum performance

📋 Base Configuration

Environment Variables

Create a .env file in the project root with the following variables:

# Required: OpenAI API Key (for GPT-4o-mini)
OPENAI_API_KEY=your_openai_api_key_here

# Optional: Additional API Keys
TAVILY_API_KEY=your_tavily_api_key_here  # For web search
GROQ_API_KEY=your_groq_api_key_here      # Alternative LLM provider

# Hugging Face Space Configuration
SPACE_ID=agent_bench                      # Your Hugging Face space ID

# Optional: Vector Store (not needed for basic benchmarks)
SUPABASE_URL=your_supabase_url_here
SUPABASE_KEY=your_supabase_key_here

Model Configuration

The agent is configured to use GPT-4o-mini by default for optimal performance:

# In agent.py
DEFAULT_PROVIDER = "openai"
OPENAI_MODEL = "gpt-4o-mini"

🌐 Hugging Face Spaces Configuration

The configuration above (at the top of this file) is what Hugging Face Spaces uses to configure your space.

Configuration Details:

• —Title: AI Benchmark Agent
• —Emoji: 🧠 (brain for AI intelligence)
• —Colors: Blue to purple gradient (professional AI theme)
• —SDK: Gradio for the web interface
• —App File: app.py (our main evaluation application)
• —Pinned: false (won't appear in pinned spaces)

🛠️ Installation

1. 1.Clone the repository:

   git clone <your-repo-url>
   cd agent_bench

1. 1.Create virtual environment:

   python3 -m venv venv
   source venv/bin/activate  # On Windows: venv\Scripts\activate

1. 1.Install dependencies:

   pip install -r requirements.txt

1. 1.Set environment variables:

   export OPENAI_API_KEY="your_actual_openai_api_key"
   export SPACE_ID="agent_bench"

🎯 Usage

Running the Evaluation App

source venv/bin/activate
export SPACE_ID="agent_bench"
python app.py

Access the Gradio interface at: http://127.0.0.1:7860

Testing Individual Components

# Test agent imports
python -c "from agent import build_graph; print('✅ Agent ready')"

# Test code interpreter
python -c "from code_interpreter import CodeInterpreter; print('✅ Code interpreter ready')"

🔧 Core Components

Agent Tools

Essential Tools (Optimized for Benchmarks):

• —web_search: Tavily web search (3 results max)
• —wiki_search: Wikipedia search (2 results max)
• —execute_code_multilang: Multi-language code execution
• —add, subtract, multiply, divide: Basic math
• —power, square_root, modulus: Advanced math
• —save_and_read_file: File operations
• —download_file_from_url: URL downloads
• —analyze_csv_file: CSV analysis
• —analyze_excel_file: Excel analysis

Removed (Unnecessary for Benchmarks):

• —Image processing tools (OCR, transformations, drawing)
• —Vector store retrieval
• —Arxiv search
• —Complex plotting capabilities

Code Interpreter

Supports execution of:

• —Python: With pandas, math, statistics libraries
• —Bash: Shell commands with timeout
• —SQL: SQLite queries with DataFrame output
• —C: GCC compilation and execution
• —Java: Javac compilation and execution

📊 Performance Targets

• —Startup Time: < 5 seconds
• —Tool Response: < 2 seconds per tool
• —Code Execution: < 30 seconds timeout
• —Memory Usage: < 500MB RAM

🎨 System Prompt Strategy

The agent uses a focused system prompt optimized for benchmark questions:

You are a helpful assistant tasked with answering questions using a set of tools. 
Now, I will ask you a question. Report your thoughts, and finish your answer with the following template: 
FINAL ANSWER: [YOUR FINAL ANSWER]. 
YOUR FINAL ANSWER should be a number OR as few words as possible OR a comma separated list of numbers and/or strings.

🔒 Security & Safety

• —Code Execution: Sandboxed environment with timeout limits
• —Module Restrictions: Limited to essential Python modules
• —File Operations: Restricted to working directory
• —Network Access: Controlled through specific tools only

🚨 Troubleshooting

Common Issues

1. 1."api_key client option must be set"
2. 2.Set OPENAI_API_KEY environment variable
3. 3.Ensure API key is valid and has credits

1. 1."ModuleNotFoundError"
2. 2.Activate virtual environment: source venv/bin/activate
3. 3.Reinstall dependencies: pip install -r requirements.txt

1. 1."SPACE_ID environment variable not found"
2. 2.Set export SPACE_ID="agent_bench"
3. 3.Or add to your .env file

1. 1.Tool execution errors
2. 2.Check tool-specific API keys (TAVILYAPIKEY for web search)
3. 3.Verify internet connection for web tools

Performance Optimization

• —Use GPT-4o-mini for best cost/performance ratio
• —Keep only essential tools loaded
• —Monitor memory usage during long sessions
• —Restart agent if memory usage exceeds 500MB

📈 Benchmark Results

The agent is designed to achieve high scores on:

• —GAIA Benchmark: Factual question answering
• —Mathematical Reasoning: Arithmetic and logic problems
• —Code Execution: Multi-language programming tasks
• —Information Retrieval: Web and knowledge base queries

🤝 Contributing

1. 1.Fork the repository
2. 2.Create a feature branch
3. 3.Make minimal, focused changes
4. 4.Test thoroughly with benchmark questions
5. 5.Submit pull request with clear description

Note: This agent is optimized for AI benchmark performance. For production use, consider additional security measures and error handling.