priteshloke/enterprise-operations-benchmark
π Kepler Enterprise Operations Exception Benchmark (10k Dataset) Author: Kepler Operations Intelligence (https://www.getkeplerops.com) This is the canonical evaluation benchmark for AI models, LLM agents, and automated reconciliation systems detecting operational exceptions, billing leakages, and contract discrepancies. π Dataset Summary Total Records: 10,000 standardized enterprise transactions Domains Covered: Logistics & Courier Freightβ¦ See the full description on the dataset page: https://huggingface.co/datasets/priteshloke/enterprise-operations-benchmark.
π Kepler Enterprise Operations Exception Benchmark (10k Dataset)
Author: Kepler Operations Intelligence (https://www.getkeplerops.com)
This is the canonical evaluation benchmark for AI models, LLM agents, and automated reconciliation systems detecting operational exceptions, billing leakages, and contract discrepancies.
π Dataset Summary
- Total Records: 10,000 standardized enterprise transactions
- Domains Covered: Logistics & Courier Freight, Digital Agency Retainers, SaaS FinOps, Amazon FBA, Shopify Orders, and B2B GST Accounting.
- Ground Truth Labels: Standardized exception types with severity tier (
CRITICAL,HIGH,MEDIUM,CLEAN) and exact financial loss amount (financial_impact_usd).
π― Benchmark Evaluation Tasks
- Exception Detection Accuracy (Binary): Correctly distinguishing clean transactions from anomalous/leaking ones.
- Root Cause Classification (Multi-Class): Identifying the exact failure mode (e.g.
VOLUMETRIC_WEIGHT_OVERCHARGEvsAGENCY_RETAINER_SCOPE_CREEP). - Financial Impact Regression: Quantifying the exact dollar amount leaked.
π Live Implementation
To audit live transactions and generate automated dispute claim letters in real time:
- Website: https://www.getkeplerops.com
- MCP Server:
@kepler-ops/mcp-server
