vibrantlabsai/enterprise-worlds

<p align="center"> <img src="https://huggingface.co/datasets/vibrantlabsai/enterprise-worlds/resolve/main/assets/social-card.png" alt="Enterprise-Worlds — RL environments for enterprise work" width="100%"> </p>

Enterprise-Worlds

Executable enterprise environments for measuring agents on operational work — the layer that turns an agent's output into a durable outcome. Each world ships persistent state, typed tools, a written policy, and a simulated colleague who discloses information only when asked. Reward is read from the final state of the world, not from the transcript.

This repository hosts the datasets. The environment and evaluator live at vibrantlabsai/Enterprise-Worlds.

Subsets

from datasets import load_dataset

tasks = load_dataset("vibrantlabsai/enterprise-worlds", "itsm", split="test")
runs  = load_dataset("vibrantlabsai/enterprise-worlds", "itsm_trajectories", split="train")

The seed worlds and the policy are plain files under itsm/world/ — they are the environment, not tabular data:

ITSMBench

IT Service Management sits on a company's control surface: identity, access, incidents, change, approvals, ownership. A one-line request ("deactivate this account") expands into a policy-governed cascade — find the affected records, separate the mandatory consequences from the ones needing the operator's authorization, and leave everything else alone.

Each task specifies a persona with private known_info, a natural-language goal, and evaluation criteria: a gold tool-call sequence replayed to produce the expected final database state, plus natural-language assertions for what the database cannot express. 31 tasks are cross-org (MSP), 22 single-tenant. The action space is 93 typed tools.

itsm columns

task_id, tenancy, seed_db, org_ids, current_time, operator_* (the acting user), personality, role_description, known_info, task_description, simulator_guidance, gold_actions, num_gold_actions, nl_assertions, initial_state_delta, raw.

Free-form nested fields (known_info, gold_actions, initial_state_delta) are JSON strings — their shapes vary per task. raw is the complete original task object, so json.loads(row["raw"]) round-trips exactly to the environment's own format.

itsm_trajectories columns

model, task_id, trial, reward, db_match, mismatches, num_mismatches, text_judgments, nl_scored, stopped, num_tool_calls, num_messages, num_tool_errors, run_error, trajectory.

trajectory is the full conversation: each message carries role, content, tool_calls (with arguments as a JSON string), and tool_error. mismatches are per-field diffs against the expected final state, e.g. incident_sla/TSLA_008 stage: 'in_progress' != 'paused'.

Provenance and license

The ITSM seed databases and typed tool surface are adapted from EnterpriseOps-Gym (Apache-2.0):

EnterpriseOps-Gym: Environments and Evaluations for Stateful Agentic Planning and Tool Use in Enterprise Settings Shiva Krishna Reddy Malay et al. (ServiceNow) — arXiv:2603.13594

Original to this work: the benchmark tasks, the multi-turn conversational user simulator, the org-scoped multi-tenant task model, and a verifier combining state-based database comparison with natural-language assertions.

Released under Apache-2.0.

@misc{enterpriseworlds2026,
      title={{Enterprise-Worlds: Executable Enterprise Environments for Measuring Operational Agents}},
      author={Shahul Elavakkattil and Ankit Sridhar and Andrew Bastin and Jithin James and Kumar Anirudha and Arjun Devarajan},
      year={2026},
      publisher={Vibrant Labs},
      url={https://github.com/vibrantlabsai/Enterprise-Worlds},
}