epiq-ai-labs/epiq-laer-corporate-benchmark

Epiq LAER CorporateBench

Harbor release of Epiq LAER CorporateBench: Enterprise Knowledge.

[image]

Epiq LAER CorporateBench is CorporateBench in its Harbor configuration. It packages the five CorporateBench capabilities as 128 scored Harbor tasks with 1,132 graded cases over the four synthetic companies of the original paper, with five development tasks alongside.

What is this?

LLMs are increasingly able to answer complex questions about enterprise-scale document collections. But evaluation is hard: companies don't want to share internal communications, and synthetic datasets have been overly simple. We present CorporateBench (CB), a human-validated multi-task Q&A benchmark whose scale approaches the conditions LLMs encounter in corporate communication networks, with evaluation corpora surpassing 230,000 documents. CB evaluates LLMs across two dimensions (information extraction and knowledge base querying) through four synthetically generated firms ranging from 12 to 10,000 employees. Each corpus is sampled from a temporally evolving knowledge base describing a consistent world, guaranteeing cross-document logical consistency even across hundreds of thousands of documents. CB provides LLM developers a metric for corporate communication reasoning, filling a crucial gap in the benchmarking ecosystem.

Dataset structure

dev/                # Five development tasks, one per capability
├── dataset.toml
├── metric.py
└── <task>/
test/               # The 128 scored tasks
├── dataset.toml
├── metric.py
└── <task>/
registry.json       # Harbor registry entries for both splits

Each task folder is a standard Harbor task:

Company configurations

Tasks and evaluation

The three QA tasks run every question twice, once over the company's documents and once over the cleaned reference records. Grading is deterministic and uses no model judge.

A QA task's reward is the mean of its six case rewards. The headline score is 100 times the mean of the 20 company and capability cells, so every company carries 25 points and every capability carries 20.

Dataset Construction Method

1. 1.Define an RDF ontology for companies, departments, teams, employees, projects, tasks, meetings, and temporal relationships.
2. 2.Generate a valid organizational hierarchy, then simulate tasks, meetings, and hierarchy changes over a 90-day quarter.
3. 3.Annotate entities and serialize the world as an ontology-validated N-Quads graph.
4. 4.Generate corporate documents around graph-derived evidence strings. Email structure is modeled on the Enron corpus; topics, employee personalities, and formality levels provide variation.
5. 5.Derive QA answers through manually verified SPARQL queries rather than model generation.

Responsible use

CorporateBench is intended for research and evaluation of retrieval, extraction, and reasoning systems. It contains synthetic organizations and communications, not real corporate records. Generated names may nevertheless coincide with real people by chance.

Cite us

@inproceedings{hamilton2026corporatebench,
  title = {{CorporateBench}: Large-Scale {Q\&A} Benchmarking with Temporal Knowledge Bases},
  author = {Hamilton, Sil and Sun, Albert Yu and Romero, Oscar J. and Henneking, Carl-Leander and Mimno, David and Yang, Bishan and Labutov, Igor},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2026},
  publisher = {Association for Computational Linguistics},
  address = {Budapest, Hungary},
  month = {October},
  year = {2026}
}