Manasa2345/AgentBenchAI

AgentBench AI

Multi-Modal Agentic AI system that accepts documents (PDF / CSV / TXT / Image), detects the business domain, runs 4 specialized agents in parallel, applies Human-in-the-Loop (HITL) approval, and recommends the Top-2 RAG architecture combinations for that document.

This is a working prototype implementation of the case study "AgentBench AI — Multi-Modal Agentic AI System for Automated RAG Architecture Recommendation."

1. Architecture

User Upload → Input Router → Domain Detection → Parallel Agents:
                                                   1. Embedding Agent
                                                   2. Chunking Agent
                                                   3. Tokenizer Agent
                                                   4. Indexing Agent
                                → HITL Approval → Consensus Engine
                                → Top-2 Recommendation
                                → LangSmith (optional tracing)
                                → HF Spaces (deployment)

Supported domains (10): Healthcare, Banking, Insurance, Manufacturing, Retail, Telecom, Legal, Education, Pharma, Aerospace.

Multi-modal ingestion:

Agent search spaces:

Consensus scoring (per the case study weighting):

Final Score = 0.4 * Retrieval + 0.2 * Latency + 0.2 * Human Approval + 0.2 * Accuracy

2. How the prototype works

1. 1.Domain detection — a keyword-frequency classifier runs offline by default. Ticking "Refine domain detection with Groq LLM" and providing a Groq API key calls a Groq-hosted model to double-check the classification. You can provide keys two ways:
2. 2.Paste them directly into the "Optional API keys" accordion in the UI (session-only, not persisted anywhere) — the simplest way to test this on a Space you just deployed.
3. 3.Set GROQ_API_KEY / LANGCHAIN_API_KEY as repository secrets in Space Settings → this is the recommended approach for a public Space, since anyone visiting a UI textbox could otherwise see/replace what's typed there. The UI fields simply override the env vars for that session if filled in.
4. 4.Parallel agents — the 4 agents run concurrently via concurrent.futures.ThreadPoolExecutor, each scoring its candidate options against the document's characteristics (domain, modality, length) with a deterministic, seeded scoring function. This keeps the demo fast and reproducible without requiring GPU access or downloading multi-gigabyte embedding models — swap in real model calls in embedding_agent, chunking_agent, etc. for a production deployment.
5. 5.HITL approval — each agent's top pick is shown with a checkbox. Rejecting any agent lowers the "Human Approval" score component used by the Consensus Engine (mirrors "Else → Re-run search/evaluation" in the spec).
6. 6.Consensus Engine — builds combinations from each agent's top candidates, scores every combination with the weighted formula above, and returns the top 2 distinct combinations.
7. 7.Observability — if LANGCHAIN_API_KEY is set, the parallel-agent run is traced via LangSmith (@traceable). Without it, tracing is a no-op and the app runs normally.

3. Project structure

agentbench/
├── app.py                  # Gradio application (single-file prototype)
├── requirements.txt
├── Dockerfile
├── .env.example
├── dummy_dataset/
│   ├── sample_healthcare.txt
│   └── sample_retail.csv
├── evaluation_report.md
└── README.md

4. Local setup

python -m venv venv
source venv/bin/activate         # Windows: venv\Scripts\activate
pip install -r requirements.txt

cp .env.example .env             # optional — fill in GROQ_API_KEY / LANGCHAIN_API_KEY
python app.py

Open http://localhost:7860, upload one of the files in dummy_dataset/ (or your own PDF/CSV/TXT/image), click Run Domain Detection + Parallel Agents, review/approve each agent's pick, then click Run Consensus Engine.

Note: OCR on images requires the system package tesseract-ocr to be installed (see Dockerfile). Without it, images are still accepted but text extraction falls back to a placeholder.

5. Docker

docker build -t agentbench-ai .
docker run -p 7860:7860 --env-file .env agentbench-ai

6. Deploy to Hugging Face Spaces

1. 1.Create a new Space → SDK: Docker.
2. 2.Push this repo's contents to the Space's git remote:

   git init
   git remote add space https://huggingface.co/spaces/<your-username>/agentbench-ai
   git add .
   git commit -m "Initial AgentBench AI deployment"
   git push space main

1. 1.In the Space's Settings → Repository secrets, add GROQ_API_KEY and/or LANGCHAIN_API_KEY if you want those integrations enabled.
2. 2.The Space will build the Dockerfile and expose the Gradio UI on port 7860 automatically.

Alternatively, choose SDK: Gradio on a new Space and upload just app.py + requirements.txt (skip the Dockerfile) — HF Spaces will manage the environment for you.

7. Success metrics (from the case study)

See evaluation_report.md for a template to log real evaluation runs.

8. Extending this prototype

• —Replace the seeded heuristic scoring in embedding_agent / chunking_agent / tokenizer_agent / indexing_agent with real calls to sentence-transformers, a live FAISS/HNSW/Chroma benchmark, and an actual tokenizer library.
• —Wrap the agent functions as LangGraph nodes for a true graph-based orchestration (the current ThreadPoolExecutor approach mirrors LangGraph's parallel-branch behavior without the extra dependency).
• —Expose process_upload / run_consensus as FastAPI endpoints if you need a headless API in addition to the Gradio UI.
• —Persist HITL decisions and consensus results to a database for longitudinal evaluation reporting.