CoolFace
Apppublic

Tornadosky/Final_Assignment_Template

sourceHugging Faceupdated 1y agoView on Hugging Face
0likes
App README

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference

GAIA Benchmark Agent

This repository contains an enhanced agent designed to perform well on the GAIA benchmark. The goal is to achieve a score of 30% or higher on a subset of GAIA Level 1 questions.

Project Structure

  • —api.py: Main file that is used to run gradio app and start the whole testing and submission loop
  • —api_utils.py: Functions for interacting with the GAIA benchmark API
  • —tool_implementations.py: Enhanced implementations of tools for web search, calculation, image analysis
  • —answer_formatter.py: Specialized formatter to ensure answers match expected formats
  • —final_integration.py: Main script that integrates everything and provides a Gradio interface

Setup and Usage

1. Install Dependencies

bash
pip install -r requirements.txt

2. Set API Keys

You'll need to set up the following API keys:

  • —OpenAI API Key: Required for the language model and image analysis
  • —SerpAPI Key: Optional but recommended for web search capability
  • —Anthropic API Key: Optional if you want to use Claude models

These can be set as environment variables, added to a .env file (see .env-example), or added directly in the Gradio interface.

3. Run the Application

bash
python final_integration.py

This will launch a Gradio interface where you can:

  • —Configure your agent
  • —Test on individual questions
  • —Run batch tests
  • —Submit your full results to the GAIA benchmark

Enhancement Strategies

This agent uses several strategies to achieve good performance:

  1. 1.Structured Tool Usage: Makes efficient use of web search, image analysis, and calculation tools
  2. 2.Careful Answer Formatting: Ensures answers match the expected format exactly
  3. 3.Robust Error Handling: Implements retries and failure recovery
  4. 4.Effective Prompting: Uses detailed prompts with specific instructions

Testing and Debugging

Before submitting your full results, it's recommended to:

  1. 1.Test individual questions to ensure the agent is working correctly
  2. 2.Run batch tests on a small subset of questions
  3. 3.Review the answers for formatting issues

Submission Process

  1. 1.Fill in your Hugging Face username
  2. 2.Add your agent code URL (URL to this repository)
  3. 3.Set your API keys
  4. 4.Click "Process All Questions and Submit"

Further Improvements

To potentially improve your score further:

  1. 1.Fine-tune prompt engineering for specific question types
  2. 2.Implement more specialized tools for particular tasks
  3. 3.Add more sophisticated answer formatting for complex response types
  4. 4.Experiment with different LLM models and parameters

Troubleshooting

Common issues:

  • —API key errors: Ensure all required keys are set correctly
  • —Answer formatting: Review the GAIA requirements for exact matching
  • —Tool failures: Check logs for specific error messages

If you encounter issues, check the error message in the output panel or review the logs.