zade-frontier/andrej-karpathy-llm-council

🏢 LLM Council - Multi-Model AI Deliberation System

A sophisticated system where multiple LLMs collaboratively answer questions through a 3-stage deliberation process.

🎯 How It Works

1. 1.Stage 1 - Individual Responses: 5 different AI models independently answer your question
2. 2.Stage 2 - Peer Review: Each model ranks the anonymized responses from others
3. 3.Stage 3 - Synthesis: A chairman model synthesizes the final answer based on all inputs

💰 Cost: Mostly FREE!

This version uses FREE HuggingFace Inference API models:

• —✅ Meta Llama 3.3 70B (FREE)
• —✅ Qwen 2.5 72B (FREE)
• —✅ Mixtral 8x7B (FREE)
• —💵 OpenAI GPT-4o-mini (low cost)
• —💵 OpenAI GPT-3.5-turbo (low cost)

Cost per query: ~$0.01-0.03 (mostly OpenAI, HF is free!)

⚡ Quick Start

🚀 Deploy to Hugging Face (Recommended)

1. 1.Fork/Duplicate this Space
2. 2.Add your API keys in Settings → Repository secrets:
3. 3.OPENAI_API_KEY - Get from OpenAI
4. 4.HUGGINGFACE_API_KEY - Get from HuggingFace
5. 5.Done! Your space will auto-deploy

💻 Run Locally

# Clone repository
git clone https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME
cd YOUR_SPACE_NAME

# Install dependencies
pip install -r requirements.txt

# Create .env file with your API keys
cp .env.example .env
# Edit .env and add:
#   OPENAI_API_KEY=your_openai_key
#   HUGGINGFACE_API_KEY=your_hf_token

# Run the app
python app.py

Visit http://localhost:7860

🔑 Getting API Keys

OpenAI API Key (Required)

1. 1.Go to https://platform.openai.com/api-keys
2. 2.Create new secret key
3. 3.Copy and save it (costs ~$0.01-0.03 per query)

HuggingFace Token (Required for FREE models)

1. 1.Go to https://huggingface.co/settings/tokens
2. 2.Create new token (read access is enough)
3. 3.Copy and save it (100% FREE to use!)

🤖 Council Models

Current Configuration

• —Meta Llama 3.3 70B - Excellent reasoning, FREE
• —Qwen 2.5 72B - Strong performance, FREE
• —Mixtral 8x7B - Mixture of experts, FREE
• —OpenAI GPT-4o-mini - Fast & capable, low cost
• —OpenAI GPT-3.5-turbo - Reliable, low cost

Chairman

• —OpenAI GPT-4o-mini - Excellent synthesis capabilities

Want to customize? Edit backend/config_free.py!

📊 Performance

• —Response Time: 60-120 seconds (3 stages, parallel processing)
• —Quality: Better than single-model responses
• —Cost: ~$0.01-0.03 per query (mostly FREE!)
• —Reliability: Automatic retries & error handling

🛠️ Tech Stack

• —Frontend: Gradio 6.0+ (with MCP server support)
• —Backend: Python async/await
• —APIs:
• —HuggingFace Inference API (FREE models)
• —OpenAI API (paid models)
• —Storage: JSON-based conversation persistence

📁 Project Structure

llm_council/
├── app.py                     # Main Gradio interface
├── requirements.txt           # Python dependencies
├── .env.example              # Environment template
├── backend/
│   ├── config_free.py        # FREE model configuration
│   ├── api_client.py         # HF + OpenAI API client
│   ├── council_free.py       # 3-stage orchestration
│   ├── storage.py            # Conversation storage
│   └── main.py               # FastAPI backend (optional)
└── docs/
    ├── QUICKSTART.md
    ├── DEPLOYMENT_GUIDE.md
    └── CODE_ANALYSIS.md

🔧 Configuration

Want different models? Edit backend/config_free.py:

# Use ALL FREE models (no OpenAI cost):
COUNCIL_MODELS = [
    {"id": "meta-llama/Llama-3.3-70B-Instruct", "provider": "huggingface"},
    {"id": "Qwen/Qwen2.5-72B-Instruct", "provider": "huggingface"},
    {"id": "mistralai/Mixtral-8x7B-Instruct-v0.1", "provider": "huggingface"},
    {"id": "google/gemma-2-27b-it", "provider": "huggingface"},
]

🤝 Contributing

Improvements welcome! See CODE_ANALYSIS.md for refactoring suggestions.

📝 Credits

• —Implementation: Community contributions
• —FREE models: Meta, Qwen, Mistral via HuggingFace

📄 License

See original repository for license information.

Need Help? Check the docs folder for detailed guides!