CoolFace
Apppublic

MCP-1st-Birthday/commerce-shopping-agent

sourceHugging Facemitupdated 10mo agoView on Hugging Face
1likes
App README

Commerce Shopping Agent

AI-powered shopping agent with input/output guardrails for safe interactions.

Features

  • —🛒 Natural language product search
  • —💰 Multi-retailer price comparison
  • —📊 Detailed product information
  • —🛡️ Input/Output Guardrails using GuardrailsAI framework
  • —🤖 Powered by LangGraph + MCP

Guardrails

This agent includes comprehensive guardrails for both input and output:

Input Guardrails

  • —Toxic Language Detection: Filters inappropriate language
  • —PII Detection: Detects and redacts personally identifiable information
  • —Jailbreak Detection: Prevents prompt injection attacks
  • —Topic Restriction: Ensures conversations stay on-topic (shopping, products, e-commerce)
  • —Competitor Check: Optional detection of competitor mentions

Output Guardrails

  • —Toxic Language Detection: Ensures responses are appropriate
  • —PII Redaction: Removes sensitive information from responses
  • —Topic Compliance: Ensures responses stay relevant
  • —Reading Time Check: Prevents overly long responses

Setup

  1. 1.Install dependencies:
bash
pip install -r requirements.txt
  1. 1.Install GuardrailsAI validators (if needed):
bash
guardrails hub install hub://guardrails/toxic_language
guardrails hub install hub://guardrails/detect_pii
guardrails hub install hub://guardrails/detect_jailbreak
guardrails hub install hub://guardrails/restrict_to_topic
  1. 1.Configure environment variables (see .env.example)
  1. 1.Run the application:
bash
python app.py

Configuration

Environment Variables

ChromaDB Cloud (Optional):

  • —CHROMA_API_KEY: Your ChromaDB Cloud API key
  • —CHROMA_TENANT: Your ChromaDB Cloud tenant ID
  • —CHROMA_DATABASE: Your ChromaDB Cloud database name

If these are set, the app will use ChromaDB Cloud for persistent memory storage. Otherwise, it uses local storage.

HuggingFace OAuth (for HuggingFace Spaces):

  • —OAuth works automatically in HuggingFace Spaces
  • —For local development, you may need to set HF_TOKEN or run huggingface-cli login

Guardrails: Guardrails can be configured in src/guardrails.py:

  • —Enable/disable input/output guardrails
  • —Customize topic restrictions
  • —Configure PII entities to detect
  • —Set competitor lists

Testing

The project includes comprehensive tests for guardrails functionality.

Install Test Dependencies

bash
pip install -e ".[dev]"
# or
pip install pytest pytest-asyncio pytest-mock

Run Tests

Run all tests:

bash
pytest

Run with verbose output:

bash
pytest -v

Run only guardrails tests:

bash
pytest tests/test_guardrails.py

Run memory tests:

bash
pytest tests/test_memory.py

Run memory example script (standalone):

bash
python -m pytest tests/test_memory.py::test_memory_script_example -v -s
# or run directly:
python tests/test_memory.py

Run integration tests:

bash
pytest -m integration

Run specific test class:

bash
pytest tests/test_guardrails.py::TestInputValidation
pytest tests/test_memory.py::TestMemoryManager

Test Coverage

The test suite includes:

  • —✅ Input validation tests (valid/invalid inputs, PII handling, edge cases)
  • —✅ Output validation tests (valid/invalid outputs, PII redaction)
  • —✅ Configuration tests (enabled/disabled guardrails, custom settings)
  • —✅ Error handling tests (exception handling, fallback responses)
  • —✅ Integration tests (app and shopping agent integration)
  • —✅ Edge case tests (empty strings, unicode, special characters)
  • —✅ Mock tests (guardrails-ai unavailable scenarios)
  • —✅ Memory tests (storage, retrieval, semantic search, filtering)
  • —✅ Memory integration tests (agent memory capture and retrieval)

Evaluations

The project includes two evaluation frameworks to measure agent performance:

1. LangSmith Evaluations

Custom evaluators with LangSmith integration for tracking and analysis.

Setup:

bash
export LANGSMITH_API_KEY=your_api_key_here
export LANGSMITH_PROJECT=commerce-shopping-agent-evals

Run:

bash
python -m evals.evaluate_agent

Metrics:

  • —Relevance, Completeness, Correctness, Safety

2. Ragas Evaluations

Specialized agent metrics using the Ragas framework for AI agent evaluation.

Setup:

bash
export OPENAI_API_KEY=your_openai_key  # Required for evaluator LLM

Run:

bash
python -m evals.evaluate_agent_ragas

Metrics:

  • —Agent Goal Accuracy: Measures if the agent achieved the user's goal
  • —Topic Adherence: Ensures the agent stays on topic
  • —Tool Call Accuracy: Validates correct tool usage
  • —Tool Call F1: Measures precision and recall of tool calls

Ragas provides LLM-as-a-judge metrics specifically designed for evaluating AI agents with tool use capabilities.

See evals/README.md for detailed documentation on both evaluation frameworks.

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference