Apppublic

navamara55/callcenter-agent

sourceHugging Facemitupdated 8mo agoView on Hugging Face

0likes

App README

AI Call Center Assistant 📞

Multi-Agent AI System for Call Center Audio Analysis with Automated Transcription, Summarization, Sentiment Analysis, and Quality Scoring.

![Python 3.10+](https://www.python.org/downloads/) ![License: MIT](https://opensource.org/licenses/MIT) ![Streamlit](https://streamlit.io)

🚀 [Live Demo](https://huggingface.co/spaces/navamara55/callcenter-agent) - Try it on Hugging Face Spaces!

🌟 Features

—🎙️ Audio Transcription: Automatic speech-to-text using OpenAI Whisper API with intelligent caching
—📝 Call Summarization: Concise, structured summaries with key points and resolutions
—💭 Sentiment Analysis: Customer and agent sentiment tracking with emotional journey mapping
—📊 Quality Scoring: Comprehensive evaluation on 5 criteria (greeting, empathy, resolution, professionalism, adherence)
—✅ Action Item Extraction: Automatic identification of follow-ups, commitments, and tasks
—👥 Role-Based Views: Specialized dashboards for Agents, Managers, and QA Teams
—📈 Interactive Visualizations: Radar charts, sentiment timelines, and exportable action items
—� PDF Export: Professional reports with text and visual charts for stakeholder distribution
—�🔄 Conversation Memory: Context-aware interactions using LangGraph checkpointing
—⚡ Smart Routing: GPT-4o-mini powered query classification and agent routing
—🛡️ Production Guardrails: Rate limiting, cost control, circuit breaker, and metrics tracking

📸 Screenshots

Home View

[image]

Agent View

[image]

Manager View

[image]

Quality Score Dashboard

[image]

Call Summary

[image]

Topic Tags

[image]

🏗️ Architecture

Multi-Agent System with Guardrails

┌─────────────────────────────────────────────────────────┐
│                    User Query                            │
└─────────────────────┬───────────────────────────────────┘
                      │
              ┌───────▼────────┐
              │  Query Router  │ (GPT-4o-mini)
              └───────┬────────┘
                      │
        ┌─────────────┼─────────────┐
        │             │             │
   ┌────▼───┐   ┌────▼────┐   ┌───▼─────┐
   │ Call   │   │ Trans-  │   │ Summar- │
   │ Intake │   │ cription│   │ ization │
   └────────┘   └────┬────┘   └─────────┘
                     │
            ┌────────┼────────┐
       ┌────▼────┐      ┌────▼────────┐
       │ Quality │      │  Sentiment  │
       │ Scoring │      │  Analysis   │
       └─────────┘      └─────────────┘

═══════════════════════════════════════════════════════════
         🛡️ Guardrail Layer (All API Calls)
═══════════════════════════════════════════════════════════
  Rate Limiting │ Cost Tracking │ Circuit Breaker │ Metrics
═══════════════════════════════════════════════════════════

5 Specialized Agents

1.Call Intake Agent: Initial query handling and system guidance
2.Transcription Agent: Audio → text with speaker labeling (Whisper + GPT-4o-mini)
3.Summarization Agent: Structured call summaries with key insights
4.Quality Scoring Agent: 1-10 scoring on 5 criteria with justifications
5.Sentiment Analysis Agent: Emotion tracking + action item extraction

🚀 Quick Start

Prerequisites

—Python 3.10 or higher
—OpenAI API key
—(Optional) Hugging Face token for datasets

Installation

1.Clone the repository

bash

git clone https://github.com/yourusername/ai-callcenter-assistant.git
cd ai-callcenter-assistant

1.Install uv (if not already installed)

bash

curl -LsSf https://astral.sh/uv/install.sh | sh

Or on macOS/Linux:

bash

brew install uv

1.Install dependencies with uv (recommended)

bash

uv sync

Or using pip:

bash

python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -r requirements.txt

1.Set up environment variables

bash

cp .env.example .env

Edit .env and add your API keys:

env

OPENAI_API_KEY=sk-your-openai-api-key-here
HF_TOKEN=hf_your-huggingface-token-here  # Optional

1.Run the application

With UV (recommended):

bash

uv run python -m streamlit run app.py --server.port 8501

Or using pip/venv:

bash

source venv/bin/activate
streamlit run app.py

The app will open in your browser at http://localhost:8501

💡 Tip: See docs/UV_COMMANDS.md for all available UV commands

📖 Usage

Upload and Analyze Calls

1.Upload Audio: Click "Upload Call Recording" in the sidebar
2.Supported formats: MP3, WAV, M4A, OGG
3.Max size: 25MB

1.Transcribe: Click "🎙️ Transcribe Audio"
2.First transcription uses Whisper API
3.Subsequent transcriptions of the same file use cache

1.Analyze: Choose from three views:

👤 Agent View

—Generate call summaries
—View action items
—Quick insights for call handlers

📊 Manager View

—Quality score radar charts
—Performance metrics
—Strengths and improvement areas

🔍 QA View

—Detailed quality evaluation
—Sentiment analysis
—Action item tracking with CSV export

Example Queries

—"Transcribe this audio file"
—"Summarize this call"
—"What was the customer sentiment?"
—"Evaluate the quality of this conversation"
—"What action items were mentioned?"

🧪 Testing

Run Tests

With UV (recommended):

bash

# Run all tests
uv run pytest tests/ -v

# Run with coverage
uv run pytest tests/ --cov=src/ai_callcenter_assistant --cov-report=html

# Using test runner script
./run_tests_uv.sh all

Traditional method:

bash

pytest tests/ -v

Test Coverage

—125+ test methods across 27+ test classes
—2,500+ lines of test code
—87.5%+ passing rate (112/128 tests)
—Coverage: Agents (100%), Orchestration (95%+), Router (95%+)

See docs/testing/TESTING.md for detailed testing guide and docs/UV_COMMANDS.md for all UV test commands.

📊 Evaluation

The system uses BLEU scores to evaluate summarization quality.

Run Evaluation

bash

uv run evaluate-system

Evaluation Metrics

—BLEU Score: Measures summary quality against references
—Quality Score MAE: Mean absolute error for quality predictions
—Action Item F1: Precision and recall for action item extraction

Creating Ground Truth

1.Prepare reference summaries in data/evaluation/ground_truth.json
2.Run system on test calls
3.Compare outputs using the evaluator

python

from ai_callcenter_assistant.utils.evaluator import get_system_evaluator

evaluator = get_system_evaluator()
results = evaluator.evaluate_summaries(generated, references)
print(f"Mean BLEU: {results['mean_bleu']:.4f}")

🧩 Project Structure

ai-callcenter-assistant/
├── app.py                          # Streamlit application
├── pyproject.toml                  # Dependencies & config
├── requirements.txt                # Pinned dependencies
├── Dockerfile                      # Container definition
├── .env.example                    # Environment template
│
├── src/
│   └── ai_callcenter_assistant/
│       ├── config.py               # Configuration & prompts
│       ├── graph.py                # LangGraph workflow
│       ├── router.py               # Query routing
│       │
│       ├── agents/                 # 5 specialized agents
│       │   ├── call_intake.py
│       │   ├── transcription.py
│       │   ├── summarization.py
│       │   ├── quality_scoring.py
│       │   └── sentiment_analysis.py
│       │
│       ├── tools/                  # Core tools
│       │   └── whisper_transcriber.py
│       │
│       ├── utils/                  # Utilities
│       │   └── evaluator.py
│       │
│       └── scripts/                # CLI scripts
│           ├── download_audio_datasets.py
│           └── evaluate_system.py
│
├── tests/                          # Unit tests
│   ├── test_router.py
│   ├── test_agents.py
│   └── test_whisper.py
│
├── data/
│   ├── audio_files/                # Uploaded audio
│   ├── transcripts/                # Cached transcripts
│   └── evaluation/                 # Evaluation data
│
└── .github/
    └── workflows/
        └── deploy.yml              # HF Spaces deployment

🧪 Testing

Run All Tests

bash

uv run pytest

Run with Coverage

bash

uv run pytest --cov=src/ai_callcenter_assistant --cov-report=html

Run Specific Tests

bash

pytest tests/test_router.py
pytest tests/test_agents.py
pytest tests/test_whisper.py

🐳 Docker

Build Image

bash

docker build -t ai-callcenter-assistant .

Run Container

bash

docker run -p 8501:8501 \
  -e OPENAI_API_KEY=your-key-here \
  ai-callcenter-assistant

Using Docker Compose

bash

docker-compose up

🚀 Deployment

Hugging Face Spaces

1.Create a new Space on Hugging Face
2.Choose "Streamlit" as SDK
3.Name it ai-callcenter-assistant

1.Add Secrets in Space settings
2.OPENAI_API_KEY: Your OpenAI API key

1.Configure GitHub Actions
2.Add HF_TOKEN to repository secrets
3.Update .github/workflows/deploy.yml with your HF username

1.Push to trigger deployment

bash

git push origin main

Manual Deployment

bash

# Add Hugging Face remote
git remote add hf https://huggingface.co/spaces/YOUR_USERNAME/ai-callcenter-assistant

# Push to HF Spaces
git push hf main

⚙️ Configuration

Model Settings

Edit config.py or set environment variables:

python

OPENAI_MODEL_ROUTER = "gpt-4o-mini"    # Router model
OPENAI_MODEL_AGENT = "gpt-4o-mini"     # Agent model
WHISPER_MODEL = "whisper-1"            # Transcription model

ROUTER_TEMPERATURE = 0.0               # Deterministic routing
AGENT_TEMPERATURE = 0.3                # Creative responses

🛡️ Production Guardrails

The system includes comprehensive production guardrails for cost control and reliability:

Rate Limiting:

bash

RATE_LIMIT_RPM=60              # Requests per minute
RATE_LIMIT_TPM=90000           # Tokens per minute
MAX_TOKENS_PER_REQUEST=2000    # Max tokens per request

Cost Control:

bash

MAX_COST_PER_SESSION=1.00      # Maximum USD per session

Reliability:

bash

MAX_RETRIES=3                  # Retry attempts on failure
RETRY_BACKOFF_FACTOR=2.0       # Exponential backoff multiplier
REQUEST_TIMEOUT=30             # Request timeout (seconds)
CIRCUIT_BREAKER_THRESHOLD=5    # Failures before circuit opens
CIRCUIT_BREAKER_TIMEOUT=60     # Circuit reset timeout (seconds)

Features:

—✅ Token bucket rate limiting (thread-safe)
—✅ Automatic cost tracking and budget enforcement
—✅ Circuit breaker for cascading failure prevention
—✅ Exponential backoff retry logic
—✅ Real-time metrics and monitoring
—✅ UI dashboard with budget usage display

See docs/architecture/guardrails.md for complete documentation.

SSL/TLS Configuration (macOS/uv)

Note: This project disables SSL verification as a workaround for certificate issues with uv-managed Python on macOS. The configuration is in config.py.

For production deployments:

—Use system Python or Docker (where SSL works correctly)
—Or configure proper SSL certificates for your environment

Audio Limits

python

MAX_AUDIO_SIZE_MB = 25                 # Max upload size
MAX_AUDIO_DURATION_SECONDS = 600       # 10 minutes

Quality Criteria

python

QUALITY_CRITERIA = [
    "greeting",
    "empathy",
    "problem_resolution",
    "professionalism",
    "script_adherence"
]

💰 Cost Optimization

Caching Strategy

—Whisper transcriptions: Permanent cache using file hash (no re-transcription)
—Agent responses: Session-based cache with configurable TTL
—LangGraph memory: Efficient conversation context management

Model Selection

—Router: GPT-4o-mini ($0.15/$0.60 per 1M tokens) for classification
—Agents: GPT-4o-mini for all analysis tasks
—Whisper: $0.006/minute with aggressive caching

Estimated Costs (per call)

—Transcription (5 min): ~$0.03 (cached after first use)
—Routing: ~$0.0001
—Analysis (all agents): ~$0.01-0.02
—Total: ~$0.04 per call (first analysis), ~$0.01 (cached transcript)

🔧 Development

Setup Development Environment

bash

# Install with dev dependencies
uv sync --all-extras

# Install pre-commit hooks (if using)
uv run pre-commit install

Code Formatting

bash

uv run black src/ tests/
uv run ruff check src/ tests/ --fix

Adding New Agents

1.Create agent file in src/ai_callcenter_assistant/agents/
2.Follow existing agent pattern (inherit system prompt, process method)
3.Register in graph.py
4.Update router with new agent name
5.Add tests in tests/

📊 Performance

—Transcription: ~2-5 seconds per minute of audio (Whisper API)
—Analysis: ~3-10 seconds per query (GPT-4o-mini)
—Routing: <1 second (GPT-4o-mini)
—Total: ~10-20 seconds for full analysis of 5-minute call

🤝 Contributing

Contributions are welcome! Please follow these steps:

1.Fork the repository
2.Create a feature branch (git checkout -b feature/amazing-feature)
3.Commit your changes (git commit -m 'Add amazing feature')
4.Push to branch (git push origin feature/amazing-feature)
5.Open a Pull Request

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

—OpenAI for GPT-4o-mini and Whisper API
—LangChain and LangGraph for agent orchestration
—Streamlit for the web interface
—Hugging Face for hosting and datasets

📧 Contact

For questions, issues, or feedback:

—Create an Issue
—Email: your.email@example.com

🗺️ Roadmap

—[ ] Real-time audio streaming support
—[ ] Speaker diarization with pyannote-audio
—[ ] Multi-language support
—[ ] Custom quality scoring rubrics
—[ ] Batch processing API
—[ ] Analytics dashboard
—[ ] Integration with CRM systems

Built with ❤️ using GPT-4o-mini, OpenAI Whisper, LangGraph, and Streamlit