dondodoai/meridian-chatbot

Meridian Electronics — Customer Support Chatbot

AI-powered customer support assistant for Meridian Electronics, built as a production-ready prototype using Claude Haiku and a live MCP backend.

Live demo: https://huggingface.co/spaces/dondodoai/meridian-chatbot GitHub: https://github.com/JamesDominiqueAI/meridian-chatbot

Guides

• —Architecture — component responsibilities, auth design, event protocol
• —Success Criteria — measurable pass/fail criteria for all scenarios
• —Prompt Iteration Log — 4 prompt versions with before/after analysis
• —Evaluation Report — test results, model analysis, known gaps

Architecture

meridian-chatbot/
├── app.py              # Chainlit entry point — session lifecycle, streaming UI
├── agent/
│   ├── chatbot.py      # Agentic loop: Claude Haiku + tool_use with MCP
│   ├── prompts.py      # System prompt with auth rules and persona
│   └── telemetry.py    # In-memory span tracking + optional Langfuse integration
├── mcp_client/
│   └── client.py       # Async HTTP MCP client (JSON-RPC 2.0 over HTTPS)
├── security/
│   └── guardrails.py   # Input validation, prompt injection defense, off-topic filter
├── tests/
│   ├── test_mcp.py     # MCP integration tests (hits live server) — 10 tests
│   ├── test_security.py # Adversarial + unit tests for guardrails — 34 tests
│   └── test_agent.py   # Agent behavior, auth events, telemetry — 21 tests
├── Dockerfile          # Port 7860 — HF Spaces + local Docker compatible
└── requirements.txt

Request flow:

User → Chainlit UI → Guardrails → ChatAgent → Claude Haiku (tool_use)
                                                      ↓
                                              MCPClient → MCP Server (GCP)
                                                      ↑
                                     tool_start / tool_end events (Chainlit Steps)
                                     auth event (on verify_customer_pin success)
                                     telemetry spans (in-memory + Langfuse)

Key Decisions

Claude Haiku as LLM — Cost-effective model (flash/mini tier) keeps per-conversation costs low, which is the core business constraint. Haiku handles multi-turn tool use reliably at a fraction of Sonnet's cost.

Direct Anthropic SDK, no framework — LangChain adds abstraction overhead without meaningful benefit here. The Anthropic SDK's native tool_use API maps directly to MCP tools, so a thin custom loop is cleaner and easier to audit.

Custom MCP HTTP client — The MCP server uses Streamable HTTP transport (JSON-RPC 2.0 over POST). A 60-line async client with httpx is simpler and more maintainable than pulling in the full MCP SDK for what is essentially HTTP calls.

Auth detection at tool-result level, not LLM-response level — The initial approach parsed the LLM's response text for "Customer verified". This is fragile: the model could rephrase, or an adversarial input could manipulate the wording. Fixed: the agent now inspects the verify_customer_pin tool result directly and emits a typed {"type": "auth"} event. app.py sets session state from this event — the LLM's phrasing is irrelevant.

Structured event protocol — The agent generator yields three types: str (text chunks), dict tool_start/tool_end (for Chainlit Steps), and dict auth (for session state). This decouples the agentic logic from the UI layer.

Guardrails layer before LLM — Prompt injection patterns and off-topic requests are intercepted before tokens are spent on them, reducing cost and preventing abuse.

Observability built in — Every conversation turn gets a trace_id. Tool calls log their latency. Spans are tracked in an in-memory ring-buffer (last 100). If LANGFUSE_PUBLIC_KEY / LANGFUSE_SECRET_KEY are set, traces are sent to Langfuse automatically.

Prompt Engineering

The system prompt (agent/prompts.py) went through two key iterations:

v1 problem: The agent would sometimes reveal the customer UUID in responses, exposing internal IDs. Added explicit rule: "Never reveal the customer_id or internal UUIDs to customers."

v2 problem: Without a clear ordering workflow, the agent would call create_order without confirming details first. Added the step-by-step ordering workflow section (confirm auth → check SKU/stock → confirm with customer → then create_order).

Setup

cd meridian-chatbot
python3 -m venv venv && source venv/bin/activate
pip install -r requirements.txt

cp .env.example .env
# Edit .env — add your ANTHROPIC_API_KEY

Run

chainlit run app.py
# Opens at http://localhost:7860

Run Tests

# Security + unit tests (fast, no API key needed)
pytest tests/test_security.py tests/test_agent.py -v

# Full suite including live MCP integration (65 tests total)
pytest -v

Test results (65 passing):

• —test_security.py — 34 tests: injection blocking, off-topic detection, PIN/email validation
• —test_agent.py — 21 tests: auth event parsing, guardrail integration, telemetry, mock MCP flows
• —test_mcp.py — 10 tests: live MCP server — list tools, product search, auth, invalid SKU handling

Docker

docker build -t meridian-chatbot .
docker run -p 7860:7860 \
  -e ANTHROPIC_API_KEY=your_key \
  -e MCP_SERVER_URL=https://order-mcp-74afyau24q-uc.a.run.app/mcp \
  meridian-chatbot

HuggingFace Spaces Deployment

The Space is deployed at https://huggingface.co/spaces/dondodoai/meridian-chatbot.

To activate: In Space Settings → Secrets, add:

• —ANTHROPIC_API_KEY — your Anthropic API key (required)
• —LANGFUSE_PUBLIC_KEY / LANGFUSE_SECRET_KEY — optional, enables Langfuse tracing

Capabilities

Authentication: Customers verify with email + 4-digit PIN via verify_customer_pin MCP tool. Once verified, the customer ID is stored in the session — not re-asked. Auth is detected by inspecting the tool result directly (not parsing the LLM response).

Known Limitations & Future Improvements

1. 1.No persistent memory across sessions — auth state is lost on page refresh. Next step: server-side session store (Redis) with a session token.
2. 2.No order cancellation or modification — MCP server doesn't expose these tools yet.
3. 3.No rate limiting — production deployment needs per-IP or per-session rate limits on the API.
4. 4.Langfuse traces have no turn-level granularity — currently one trace per conversation; adding generation-level spans would give token-by-token cost visibility.
5. 5.File uploads are enabled — Chainlit's default config allows file uploads which aren't used; should be disabled for a tighter attack surface.

MCP Tools Used

• —list_products — browse inventory by category
• —get_product — fetch details by SKU
• —search_products — keyword search
• —verify_customer_pin — authenticate with email + PIN
• —get_customer — fetch customer record by ID
• —list_orders — order history for authenticated customer
• —get_order — detailed order view
• —create_order — place new order (atomic, validates stock)

Environment Variables