Adityax-07/AgentBench
<div align="center">
⚡ AgentBench
Multi-Agent Research Evaluation System
A production-structured LangGraph pipeline that benchmarks multi-agent vs single-agent LLM responses across 50 queries on 5 real evaluation metrics
<br/>
    
<br/>
</div>
What Is This?
Most AI demos show you an output and say "looks good." AgentBench actually measures it.
It runs two approaches on the same query — a single ReAct agent and a 5-node multi-agent pipeline — then scores both using a real LLM-as-judge evaluator on 5 metrics. Every result is computed, not hardcoded.
The core question it answers: Does splitting reasoning, research, analysis, and writing across specialised agents produce more trustworthy responses than one agent doing everything?
Answer: Yes — 6× lower hallucination. Here are the numbers.
Benchmark Results
Evaluated on 50 queries across GenAI, ML fundamentals, agentic AI, retrieval, and applied AI. Judged by llama-3.1-8b-instant as an independent LLM evaluator.Key Finding
The multi-agent pipeline is 6× more factually reliable (4% vs 24% hallucination). The Planner → Researcher → Analyst chain forces grounding before the Writer ever produces output — no claim goes unverified.
Single agent wins on speed and structural quality metrics for straightforward queries. The real tradeoff is latency vs trust, not latency vs quality.
Architecture
User Query
│
├── Single Agent (ReAct Loop)
│ └── llama-3.3-70b ──► Tavily Search ──► FAISS Retrieval
│ └── Response
│
└── Multi-Agent Pipeline (LangGraph StateGraph)
│
├── 1. Planner → Pydantic-structured subtasks + search queries
│ (llama-3.3-70b + structured output schema)
│
├── 2. Researcher → Parallel web search + FAISS vector retrieval
│ (llama-3.1-8b + Tavily + all-MiniLM-L6-v2)
│
├── 3. Analyst → Synthesises findings, scores confidence
│ (insight extraction + optional Python REPL)
│
├── 4. Writer → Structured markdown report with citations
│ (title, body, word count, sources)
│
└── 5. Memory → Persists session to SQLite (SQLiteDict)
(cross-session recall)
Both outputs evaluated by:
LLM-as-Judge (llama-3.1-8b-instant)
└── Relevance · Hallucination · Coherence · Completeness · DepthDashboard
Two panels in one animated Streamlit app:
Live Query — type any question, both agents respond in real time with a live pipeline progress bar. LLM-as-judge scores the responses on submission.
Benchmarks — 7 interactive Plotly charts built from real evaluated data:
Evaluation Metrics
The LLM-as-judge evaluates every (query, response) pair on:
Project Structure
AgentBench/
│
├── app.py # Streamlit dashboard (Live Query + Benchmarks)
├── graph.py # LangGraph StateGraph pipeline definition
├── evaluator.py # LLM-as-judge (5-metric scoring)
├── bench_runner.py # Resumable benchmark runner (50 queries)
├── bench_results.json # Real evaluated results
│
├── agents/
│ ├── planner.py # Query decomposition (Pydantic + llama-3.3-70b)
│ ├── researcher.py # Web search + FAISS retrieval (llama-3.1-8b)
│ ├── analyst.py # Research synthesis + confidence scoring
│ ├── writer.py # Structured report generation
│ └── single_agent.py # ReAct baseline agent
│
├── tools/
│ ├── web_search.py # Tavily search wrapper
│ ├── vector_store.py # FAISS store (all-MiniLM-L6-v2 embeddings)
│ └── python_repl.py # Sandboxed Python executor
│
├── memory/
│ └── store.py # SQLiteDict session memory
│
├── requirements.txt
└── PROBLEMS_FACED.md # 15 real engineering problems + fixesSetup
# 1. Clone the repo
git clone https://github.com/Adityax-07/AgentBench.git
cd AgentBench
# 2. Install dependencies
pip install -r requirements.txt
# 3. Add API keys
# Create a .env file with:
GROQ_API_KEY=your_groq_api_key
TAVILY_API_KEY=your_tavily_api_key
# 4. Run the dashboard
streamlit run app.py
# 5. (Optional) Run the full benchmark
python bench_runner.pyNote: The benchmark runner is resumable — if Groq's rate limit cuts it off, re-run and it picks up from where it stopped.
Tech Stack
Numbers at a Glance
50 queries evaluated 5 metrics per response 2 agents compared
6× lower hallucination 3,500+ lines of code 15 files, 7 modules
7 interactive charts 8 animated metric cards 100 LLM-judged responsesEngineering Notes
PROBLEMS_FACED.md documents 15 real engineering problems hit during development — Groq rate limit handling, LangGraph state serialization, Streamlit session resets, stream_mode format differences, and more. Each entry has the root cause, the fix, and an interview talking point.