CoolFace
Apppublic

crazycrazypete/Masters-four-Tab-OpenAI-Canary

sourceHugging Faceupdated 5mo agoView on Hugging Face
0likes
App README

Masters Telecom AI Toolkit (FastAPI + React + Tailwind)

One Docker app, multiple tools in one UI:

  • —Masters Telecom AI Knowledgebase (default): unified chat for router docs/specs, router lifecycle/replacements, POTS replacement Q&A, and Masters AI docs.
  • —Router Knowledgebase: internal router docs + datasets RAG (hybrid retrieval, citations, and clearly labeled web-assisted fallback).
  • —Telco Cost Calculator: multi-location inputs, charts, CSV import/export, PDF quote output.
  • —Routers: paste router models/SKUs to get lifecycle status (EOS/EOL) plus 4G + 5G replacement picks. Can also show antenna picks.
  • —Master's AI: Q&A over Masters Telecom reference docs (internal + external audience modes).
  • —POTS Replacement Q&A: Q&A over POTS replacement docs, with a simple preference to recommend OOMA.
  • —POTS Replacement Intake: a wizard + spreadsheet import that produces an intake ZIP (PDF summary + Excel) and stores the submission on the HF /data volume.
  • —Rapid Router: Verizon-rep order intake for routers + rate plans + quantities, with an admin (password) product loader and email submission (PDF + signature PNG).

Key operator docs:

  • —docs/architecture.md
  • —docs/runbook.md
  • —docs/hf_auth0_operator_checklist.md
  • —.env.example
  • —.codex/README.md

Environment variables

The app will run without an API key for the CSV-based matching in the Routers tool.

To enable AI/Q&A features, set:

  • —OPENAI_API_KEY
  • —OPENAI_MODEL (optional, default: gpt-5-mini)

Optional:

  • —TODAY_YYYY_MM_DD (override the date used for lifecycle status, format YYYY-MM-DD)

Unified knowledgebase / tab cutover flags:

  • —MASTERS_TOOLKIT_TAB_KNOWLEDGEBASE_ENABLED=true (new single-tab default)
  • —Legacy rollback flags (default false): MASTERS_TOOLKIT_TAB_ROUTER_KNOWLEDGEBASE_ENABLED, MASTERS_TOOLKIT_TAB_ROUTERS_ENABLED, MASTERS_TOOLKIT_TAB_MASTERS_AI_ENABLED, MASTERS_TOOLKIT_TAB_POTS_REPLACEMENT_QA_ENABLED, MASTERS_TOOLKIT_TAB_TELCO_CALCULATOR_ENABLED, MASTERS_TOOLKIT_TAB_POTS_SAVINGS_ESTIMATOR_ENABLED, MASTERS_TOOLKIT_TAB_POTS_REPLACEMENT_INTAKE_ENABLED, MASTERS_TOOLKIT_TAB_RAPID_ROUTER_ENABLED

Data file paths (normally you can ignore these):

  • —EOS_CSV_PATH default /app/routers_eos_eol_by_sku.csv
  • —DEC_CSV_PATH default /app/feb2026routers.csv
  • —PARSEC_CATALOG_PATH optional explicit Parsec catalog path (if unset, app falls back to corpus docs)

Router Knowledgebase corpus paths:

  • —ROUTER_RAG_BOOTSTRAP_ENABLED default true
  • —ROUTER_RAG_HF_DATASET_REPO default crazycrazypete/masters-telecom-kb-corpus
  • —ROUTER_RAG_HF_DATASET_REVISION default main
  • —ROUTER_RAG_HF_DATASET_SUBDIR default corpus/_RAG_Ready_KB_Organized
  • —ROUTER_RAG_BOOTSTRAP_DIR default /data/router_rag_corpus (falls back to /tmp/router_rag_corpus)
  • —ROUTER_RAG_DATA_DIR default auto-resolved from downloaded corpus (.../_RAG_Ready_KB_Organized)
  • —ROUTER_RAG_CHUNKS_PATH default <ROUTER_RAG_DATA_DIR>/04_ingestion/rag_ingestion_chunks.jsonl
  • —ROUTER_RAG_MANIFEST_PATH default <ROUTER_RAG_DATA_DIR>/03_manifests/rag_manifest_organized.csv
  • —ROUTER_RAG_CACHE_DIR default /data/router_rag_index (falls back to /tmp/router_rag_index)
  • —ROUTER_RAG_EMBED_MODEL default text-embedding-3-small
  • —ROUTER_RAG_OPENAI_MODEL default gpt-5-mini (falls back to OPENAI_MODEL if unset)
  • —ROUTER_RAG_TOP_K default 8
  • —ROUTER_RAG_MIN_SCORE default 0.18
  • —ROUTER_RAG_ENABLE_RERANK default true
  • —ROUTER_RAG_RERANK_HARD_ONLY default true (skip rerank for easy/single-fact queries)
  • —ROUTER_RAG_RERANK_HARD_MIN_TOKENS default 16
  • —ROUTER_RAG_RERANK_HARD_HINTS comma list of terms that should force rerank
  • —ROUTER_RAG_RERANK_MODEL default cross-encoder/ms-marco-MiniLM-L-6-v2
  • —ROUTER_RAG_CITATION_GATE_ENABLED default true
  • —ROUTER_RAG_CITATION_MIN_SOURCES_SPECS default 1
  • —ROUTER_RAG_CITATION_MIN_SOURCES_COMPARE default 2
  • —ROUTER_RAG_CITATION_MIN_SOURCES_REPLACEMENT default 1
  • —ROUTER_RAG_CITATION_MIN_UNIQUE_DOCS_COMPARE default 2
  • —ROUTER_RAG_WEB_MODE_DEFAULT default fast (fast or extended)
  • —ROUTER_RAG_WEB_MODE optional per-deployment override (fast or extended)
  • —ROUTER_RAG_WEB_FAST_TIMEOUT_S default 8.0
  • —ROUTER_RAG_WEB_EXTENDED_TIMEOUT_S default 18.0
  • —ROUTER_RAG_WEB_FAST_MAX_TOKENS default 900
  • —ROUTER_RAG_WEB_EXTENDED_MAX_TOKENS default 1400
  • —ROUTER_RAG_WEB_FAST_CONTEXT_SOURCES default 3
  • —ROUTER_RAG_WEB_EXTENDED_CONTEXT_SOURCES default 6
  • —ROUTER_RAG_ALIAS_DICT_PATH default <ROUTER_RAG_DATA_DIR>/03_manifests/router_aliases_dynamic.json

Unified KB performance toggles:

  • —UNIFIED_KB_CACHE_ENABLED default true
  • —UNIFIED_KB_CACHE_TTL_S default 600
  • —UNIFIED_KB_CACHE_MAX_ITEMS default 512
  • —UNIFIED_KB_ROUTER_FACT_FAST_PATH default true (fast deterministic answers from feb2026routers.csv for common spec asks)
  • —UNIFIED_KB_WEB_TIMEOUT_ROUTER_DOCS_S default 5.2
  • —UNIFIED_KB_WEB_TIMEOUT_MASTERS_S default 4.8
  • —UNIFIED_KB_WEB_TIMEOUT_POTS_S default 4.8
  • —UNIFIED_KB_WEB_TIMEOUT_EXTENDED_S default 8.0

Canonical corpus note:

  • —_RAG_Ready_KB_Organized is the canonical deployment corpus (documents + datasets + manifests + ingestion files).
  • —_RAG_Ready_KB is legacy staging and should remain untracked.

Router Knowledgebase maintenance scripts:

  • —python3 backend/scripts/router_rag_import_corpus.py --source-dir '/absolute/path/to/new_docs' --data-dir _RAG_Ready_KB_Organized --report-csv docs/reports/rag_import_report.csv --report-md docs/reports/rag_import_report.md
  • —Imports new corpus files into canonical folders using deterministic triage:
  • —includes authoritative vendor PDFs,
  • —excludes screenshots / non-ingestible artifacts / known reseller captures,
  • —dedupes by SHA-256 against existing corpus files,
  • —applies canonical naming and collision suffixing.
  • —Writes import summary reports (CSV + Markdown).
  • —Use --dry-run first to preview include/skip decisions.
  • —python3 backend/scripts/router_rag_device_gap_report.py
  • —Audits feb2026routers.csv + replacement_devices_missing_from_dec2025routers.csv against rag_manifest_organized.csv.
  • —Writes:
  • —docs/reports/router_rag_device_doc_gaps.csv
  • —docs/reports/router_rag_device_doc_gaps.md
  • —python3 backend/scripts/router_rag_rebuild_ingestion.py --data-dir _RAG_Ready_KB_Organized
  • —Rebuilds:
  • —03_manifests/rag_manifest_organized.csv
  • —04_ingestion/rag_ingestion_chunks.jsonl
  • —04_ingestion/rag_ingestion_chunks.csv
  • —04_ingestion/rag_ingestion_file_stats.csv
  • —Use --dry-run first to validate.
  • —python3 backend/scripts/router_rag_alias_growth.py --glob 'docs/evals/*.json' --out-json docs/evals/router_alias_growth_latest.json --apply-auto
  • —Mines message/transcript logs for model nicknames and typo variants.
  • —Writes alias growth report + updates dynamic alias dictionary at:
  • —_RAG_Ready_KB_Organized/03_manifests/router_aliases_dynamic.json
  • —python3 backend/scripts/router_rag_eval50.py --with-openai --semantic-grader --out docs/evals/router_rag_eval50_latest.json --md-out docs/evals/router_rag_eval50_latest.md --trend-file docs/evals/router_rag_eval50_trend.json
  • —Runs 50-case regression with rule + semantic scoring.
  • —Produces latest report + trend history used by nightly CI.
  • —python3 backend/scripts/unified_kb_eval150.py --cases docs/evals/unified_kb_eval150_cases.json --with-openai --semantic-grader --out docs/evals/unified_kb_eval150_latest.json --md-out docs/evals/unified_kb_eval150_latest.md --trend-file docs/evals/unified_kb_eval150_trend.json
  • —Runs the unified 150-case regression across router docs/lifecycle, POTS, and Masters domains.
  • —Includes rule scoring + optional second-pass semantic grading + latency tracking.
  • —Semantic grading can be scoped for speed via --semantic-policy (hard_edge_or_fail default) and --semantic-timeout-s.
  • —Supports shard runs via --start-id / --end-id (for example, run 15-case batches).
  • —Nightly workflow: .github/workflows/unified-kb-nightly-eval.yml.
  • —bash backend/scripts/run_unified_kb_eval150_chunks.sh
  • —Enforces OpenAI key precheck, runs the full 150 in 10-case shards, tracks per-question latency, and writes:
  • —docs/evals/shards10/unified_kb_eval150_<start>_<end>.json
  • —docs/evals/shards10/unified_kb_eval150_<start>_<end>.md
  • —docs/evals/shards10/unified_kb_eval150_shards10_summary.json
  • —Applies default hard web-time budgets during eval via timeout env vars (overridable).
  • —bash backend/scripts/router_rag_intake_pipeline.sh "<source_dir>"
  • —One-command intake for new corpus drops:
  • —imports/renames/dedupes (router_rag_import_corpus.py)
  • —rebuilds ingestion artifacts (router_rag_rebuild_ingestion.py)
  • —runs gap report + smoke checks (router_rag_device_gap_report.py, router_rag_smoke.py)
  • —writes timestamped reports in docs/reports/
  • —bash backend/scripts/release_gate.sh
  • —Pre-release gate (local) before push/deploy:
  • —validates auth/env configuration
  • —compiles startup-critical modules
  • —runs Router RAG smoke
  • —runs unified 150-case eval with semantic grading + strict exit
  • —enforces latency budgets (P95_BUDGET_MS, P99_BUDGET_MS)

Index caches (recommended on Hugging Face):

  • —POTS_INDEX_CACHE_DIR default /data/pots_index
  • —MASTERS_INDEX_CACHE_DIR default /data/masters_index

Hugging Face cache dirs (recommended on Hugging Face):

  • —HF_HOME default /data/hf_cache
  • —TRANSFORMERS_CACHE default /data/hf_cache/transformers
  • —SENTENCE_TRANSFORMERS_HOME default /data/hf_cache/sentence_transformers

Intake storage (recommended on Hugging Face):

  • —POTS_INTAKE_STORAGE_DIR default /data/pots_intake

Rapid Router order intake:

  • —RAPID_ROUTER_STORAGE_DIR default /data/rapid_router
  • —RAPID_ROUTER_ADMIN_PASSWORD required to enable the admin loader; store it as a hosted secret, not in repo files

Email (Microsoft 365 SMTP recommended):

  • —RAPID_ROUTER_SMTP_HOST default smtp.office365.com
  • —RAPID_ROUTER_SMTP_PORT default 587
  • —RAPID_ROUTER_SMTP_USERNAME (example: pete@pdinsights.ai)
  • —RAPID_ROUTER_SMTP_PASSWORD (mailbox password or app password, depends on tenant policy)
  • —RAPID_ROUTER_FROM_EMAIL required when SMTP/Graph email submission is enabled
  • —RAPID_ROUTER_TO_EMAIL required when SMTP/Graph email submission is enabled

OCR toggles (off by default):

  • —MASTERS_ENABLE_OCR=1
  • —POTS_ENABLE_OCR=1

Retrieval tuning:

  • —MASTERS_RAG_MIN_SCORE default 0.25 (minimum retrieval score before refusing)
  • —MASTERS_MAX_CHUNKS_PER_DOC default 4 (caps repeated chunks per document in retrieval)

Auth0 (required-auth mode):

  • —AUTH_REQUIRED=true
  • —AUTH0_DOMAIN=your-tenant.us.auth0.com
  • —AUTH0_CLIENT_ID=your_spa_client_id (optional Docker fallback for frontend build vars)
  • —AUTH0_AUDIENCE= optional. Leave unset unless your Auth0 tenant has a real custom API Identifier for this app.
  • —Do not use https://masters-toolkit-api or https://masters-toolkit-api/; those legacy placeholder values are ignored by current auth code.
  • —AUTH0_ISSUER=https://your-tenant.us.auth0.com/ (optional if AUTH0_DOMAIN is set)
  • —AUTH0_EMAIL_CLAIM=https://masterstelecom.com/email (default)
  • —AUTH_ALLOWED_EMAIL_DOMAINS=verizon.com,vzw.com,verizonwireless.com,masterstelecom.com,pdinsights.ai
  • —APP_BASE_URL=https://toolkit.masterstelecom.com (canonical app URL used for Auth0 redirect/logout targets)
  • —AUTH0_ALLOWED_CALLBACK_URLS=https://toolkit.masterstelecom.com,https://your-space.hf.space (optional safety check)
  • —AUTH0_ALLOWED_LOGOUT_URLS=https://toolkit.masterstelecom.com,https://your-space.hf.space (optional safety check)
  • —AUTH0_ALLOWED_WEB_ORIGINS=https://toolkit.masterstelecom.com,https://your-space.hf.space (optional safety check)
  • —AUTH_ALLOW_CROSS_ORIGIN_APP_BASE_URL=false (set true only if APP_BASE_URL intentionally differs from hosted runtime origin)

Hosted default:

  • —On Hugging Face Spaces, auth now defaults to required even if AUTH_REQUIRED is omitted.
  • —If Auth0 settings are incomplete or inconsistent, startup fails hard and the app does not boot in an open state.

Frontend Auth0 runtime vars (see frontend/.env.example and root .env.example):

  • —VITE_AUTH_REQUIRED=true
  • —VITE_AUTH0_DOMAIN=your-tenant.us.auth0.com
  • —VITE_AUTH0_CLIENT_ID=your_spa_client_id
  • —VITE_AUTH0_AUDIENCE= optional. Leave unset unless the deployment uses a real Auth0 API Identifier.
  • —Do not set it to https://masters-toolkit-api or https://masters-toolkit-api/.
  • —VITE_APP_BASE_URL=https://toolkit.masterstelecom.com
  • —VITE_AUTH0_DB_CONNECTION=Username-Password-Authentication
  • —VITE_ALLOWED_SIGNUP_DOMAINS=verizon.com,vzw.com,verizonwireless.com,masterstelecom.com,pdinsights.ai

Docker build behavior for Auth0 vars:

  • —Frontend auth values are generated at build time from VITE_* values.
  • —If VITE_* values are missing, Docker now falls back to backend-style vars where possible:
  • —AUTH_REQUIRED -> VITE_AUTH_REQUIRED
  • —AUTH0_DOMAIN -> VITE_AUTH0_DOMAIN
  • —AUTH0_CLIENT_ID -> VITE_AUTH0_CLIENT_ID
  • —AUTH0_AUDIENCE -> VITE_AUTH0_AUDIENCE
  • —AUTH_ALLOWED_EMAIL_DOMAINS -> VITE_ALLOWED_SIGNUP_DOMAINS
  • —APP_BASE_URL -> VITE_APP_BASE_URL

Build/debug label (shown in app header):

  • —MASTERS_TOOLKIT_BUILD_VERSION (optional)
  • —MASTERS_TOOLKIT_GIT_SHA (optional)
  • —If neither is set, the app shows startup UTC time as the build label.

API abuse protection (best-effort, in-memory):

  • —MASTERS_TOOLKIT_RATE_LIMIT_ENABLED=true
  • —MASTERS_TOOLKIT_RATE_LIMIT_WINDOW_SECONDS=60
  • —MASTERS_TOOLKIT_RATE_LIMIT_CHAT_MAX=300 (shared bucket across /api/message, /api/knowledgebase/message, /api/routers/message, /api/routers/rag/message, /api/masters/message, /api/pots/message)
  • —MASTERS_TOOLKIT_RATE_LIMIT_ADMIN_MAX=120 (bucket for /api/rapid_router/admin/*)
  • —MASTERS_TOOLKIT_RATE_LIMIT_MAX_TRACKED_KEYS=20000

Auth0 Actions to enforce allowed domains:

  • —Pre-user-registration (block signup domains not in allowlist)
  • —Post-login (block login domains not in allowlist, and add email claim to access token)

Example Post-login Action snippet:

js
exports.onExecutePostLogin = async (event, api) => {
  const allow = new Set(["verizon.com", "vzw.com", "verizonwireless.com", "masterstelecom.com", "pdinsights.ai"]);
  const email = String(event.user.email || "").toLowerCase();
  const domain = email.includes("@") ? email.split("@")[1] : "";

  if (!allow.has(domain)) {
    api.access.deny("Access restricted to approved business domains.");
    return;
  }

  api.accessToken.setCustomClaim("https://masterstelecom.com/email", email);
};

Local dev

Backend

bash
cd backend
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
uvicorn app.main:app --reload --host 0.0.0.0 --port 7860

Frontend

bash
cd frontend
npm install --no-audit --no-fund
npm run dev

Frontend dev server proxies /api to the backend.

Auth E2E (Playwright)

bash
cd frontend
npm run test:e2e:install
E2E_BASE_URL=http://127.0.0.1:7860 npm run test:e2e:auth

Optional env vars for redirect assertions:

  • —E2E_AUTH0_DOMAIN (example: masters-toolkit.us.auth0.com)
  • —PLAYWRIGHT_BASE_URL (alternate base URL variable)

RAG Quality Gate (Deterministic, no OpenAI key required)

bash
cd backend
python -m app.evals.rag_quality_gate \
  --masters-cases ../docs/evals/masters_retrieval_cases.json \
  --pots-cases ../docs/evals/pots_retrieval_cases.json \
  --response-cases ../docs/evals/rag_response_cases.json \
  --thresholds ../docs/evals/rag_quality_thresholds.json \
  --output ../docs/evals/rag_quality_latest.json \
  --force-rebuild-pots

The command exits non-zero if any configured threshold fails and writes a machine-readable report to:

  • —docs/evals/rag_quality_latest.json

Gate severity policy:

  • —Hard fail: format, grounding/source, policy, and web-assisted disclosure label checks.
  • —Warn-only (non-blocking): web-assisted relevance checks (response_web_relevance_rate_min_warn).

Backend test runner (recommended)

Run this first for a stable backend check (unit/API + Router RAG smoke):

bash
cd backend
scripts/test_backend.sh

What it does:

  • —Uses backend/.venv automatically.
  • —Runs a focused pytest suite:
  • —app/test_router_rag_module.py
  • —app/test_router_rag_api.py
  • —app/test_auth.py
  • —app/test_router_csv_sanity.py
  • —Runs Router RAG smoke queries with pass/fail checks.
  • —Forces rerank off during tests to avoid first-run model download stalls.

OpenAI-enabled mode (required key):

bash
cd backend
scripts/test_backend.sh --with-openai

If OPENAI_API_KEY is missing or still a placeholder, this exits fast with a clear error.

Seed env for local testing:

bash
cd backend
cp .env.test.example .env.test

Deploy on Hugging Face

Create a Docker Space, push this repo, and it will build and run.

This repo sets the cache env vars in the Dockerfile to point at /data. On Hugging Face, /data is a persisted volume, so your indexes won't rebuild every restart.

Gated Deploy (GitHub Actions)

Use workflow: .github/workflows/deploy-hf-gated.yml

What it enforces before deploy:

  • —Auth/security tests (app/test_auth.py, app/test_startup_rate_limit.py)
  • —Router regression tests (app/routers/router_conversation_regression_test.py, app/routers/router_tab_smoke_test.py)
  • —RAG quality gate tests + threshold evaluation

Release safety behavior:

  • —Stamps MASTERS_TOOLKIT_BUILD_VERSION and MASTERS_TOOLKIT_GIT_SHA into Hugging Face Space variables.
  • —Optional canary deployment first (if HF_SPACE_ID_CANARY is set), then production.
  • —One-click rollback via workflow_dispatch input rollback_ref.

Required repository secrets:

  • —HF_TOKEN (write access to target Spaces)
  • —HF_SPACE_ID (production Space repo id, e.g. owner/space-name)
  • —HF_SPACE_ID_CANARY (optional canary Space repo id)
  • —HF_USERNAME (optional; defaults to Space owner if unset)

Hosted validation and operator sign-off:

  • —python backend/scripts/validate_hosted_runtime.py --base-url https://your-space.hf.space --expected-build-version release-... --expected-git-sha <sha> --expect-auth-required true --expect-auth-enabled true
  • —cd frontend && ./scripts/run-hosted-smoke.sh
  • —standard post-deploy browser smoke for production and canary
  • —covers login/logout, one assistant-family query, and the POTS workspace shell
  • —docs/hf_auth0_operator_checklist.md

Post-deploy operator flow:

  1. 1.Let the gated workflow complete canary and production validation.
  2. 2.Run the hosted smoke bundle:
bash
cd frontend && ./scripts/run-hosted-smoke.sh
  1. 1.Complete docs/hf_auth0_operator_checklist.md before calling the deploy clean.