Minbyul/AgentMercury-corpus-sample

AgentMercury — corpus sample

A small, public slice of the RL training corpus used in AgentMercury: Your Agent Can Synthesize Verifiable Environments for Business Scenarios at Scale.

AgentMercury synthesizes executable worlds from high-level business scenarios and then instantiates tasks on top of them, rather than building an environment around a predefined task. Each task is an autonomous MCP tool-use investigation: a persona works inside a synthetic company's systems (CRM, email, Slack, billing, ticketing, project management, …), must discover what is wrong by reading primary records, and must write the correction back into state. Grading is not on the text of the answer — it is a set of programmatic assertions over the resulting database state.

• —📄 Paper: AgentMercury · 🤖 Models: `Minbyul/AgentMercury-Qwen3.5-4B`, `Minbyul/AgentMercury-Qwen3.5-35B-A3B` (+ -SAO variants)
• —📦 Full corpus: 43,300 tasks over 4,326 executable worlds (this repo ships 110 rows)

What is in this repo

The two files answer two different questions. tasks_100.jsonl shows how wide the corpus is (100 unrelated companies, industries and service mixes). env_detail/ shows what one world actually contains and how ten distinct tasks come out of a single environment without changing its structure — the separation between world construction and task instantiation that the paper argues for.

Row schema

{
  "id":   "shard_4/env_632967fc__seed0",     // <shard>/<env_id>__seed<N>
  "name": "env_632967fc__seed0",
  "prompt": [                                 // a single system message; no further user turn
    {"role": "system", "content": "You are acting as the following persona: …"}
  ],
  "tools": "[{\"name\": \"contracts_search\", \"description\": …, \"inputSchema\": {…}}, …]",
  "metadata": {
    "task_type": "mcp_laaj",
    "task_mode": "cud",                       // create / update / delete — the task must change state
    "task_id":  "env_632967fc__seed0",
    "env_id":   "env_632967fc",               // added in this sample; the world this task came from
    "ground_truth": {
      "task_mode": "cud",
      "rubric": "…+1 per satisfied line…",     // LLM-judge rubric
      "hard_mask": [true, false, …]            // which expected effects are *required*
    },
    "assertions": "[{\"type\": \"record_exists\", \"collection\": \"leads\", \"locate\": {…}}, …]",
    "n_expected_effects": 0,
    "arena_seed": "{\"persona\": {…}, \"services\": […], \"collections\": {…}, \"emails\": […], \"slack_messages\": […]}",
    "mcp_arena_backend": "local",              // runs in-process; no docker / e2b needed
    "persona": "…", "persona_role": "Account Manager",
    "touched_services": ["crm", "slack"],
    "af": true,                                // cross-system action risk (harder)
    "src_task_mode": "investigation"
  }
}

tools, metadata.assertions and metadata.arena_seed are JSON strings — parse them. tools is never empty; without it a rollout cannot act.

Three assertion types appear across the corpus: record_exists, record_field_equals, record_field_not_equals. This is what makes the reward deterministic — the check is whether the intended state change actually happened, not whether the model said it happened.

Composition of this sample

`tasks_100.jsonl` — 100 worlds, 100 tasks:

`env_detail/` — one world (env_632967fc), an integrated logistics company:

Note the tool-count lattice — {10, 14, 18, 22, 26} and nothing in between. Ten tools are the fixed core (employee search/list/get/update, email search/list, send-email, Slack list-channels/list-messages/post-message) and every additional resource contributes exactly four (search, list, get, update). Task size is therefore a discrete function of how many resources the world puts in front of the agent.

Quick start

from datasets import load_dataset
import json

tasks = load_dataset("Minbyul/AgentMercury-corpus-sample", "tasks_100")["train"]
row   = tasks[0]

print(row["prompt"][0]["content"][:600])       # persona + company brief + the investigation ask
tools = json.loads(row["tools"])               # tool specs the policy may call
seed  = json.loads(row["metadata"]["arena_seed"])
print(seed["persona"], seed["services"], {k: len(v) for k, v in seed["collections"].items()})
asserts = json.loads(row["metadata"]["assertions"])
print(len(asserts), "assertions:", {a["type"] for a in asserts})

# the same world, ten different tasks
env = load_dataset("Minbyul/AgentMercury-corpus-sample", "env_detail")["train"]
print(sorted(len(json.loads(r["tools"])) for r in env))    # 18,18,22,22,26,26,26,26,26,26

To actually run a task, metadata.arena_seed plus the row's tools are enough: the in-process local arena (training/code/local_arena.py in the AgentMercury code release) reconstructs the MCP services from the seed, so rollouts need no docker, no e2b, and no external server. Each rollout gets a fresh arena, so isolation is automatic.

Anonymization

Every environment in the corpus is seeded from a real-company scenario profile, but the world itself is fictional — entities, employees, customers, records and conversations are all synthesized, and no real firm's data appears in any row. In this public sample the original environment identifiers (which carried the seed firm's name as provenance) are replaced with content-independent ids of the form env_<8 hex>, and the mapping is not published. The row content was audited token-by-token against each source slug before release; the only matches are generic industry descriptors ("this is a fertilizers business"), which are part of the scenario, not an identity.

Limitations

• —All rows here are task_mode: cud / task_type: mcp_laaj — the state-changing investigation family. The sample is not stratified to reproduce the full corpus's marginals; it is 100 arbitrary worlds plus one world shown in depth.
• —n_expected_effects is 0 on many rows; grading uses assertions + ground_truth.hard_mask.
• —The 100 tasks are all seed 0, so they show between-world variation, not within-world variation. env_detail/ is there for the latter.
• —Rows come from the TRAIN split. A held-out TEST split (4,630 rows / 461 worlds) ships with the code release.

Citation

@article{jeong2026agentmercury,
  title  = {AgentMercury: Your Agent Can Synthesize Verifiable Environments for Business Scenarios at Scale},
  author = {Jeong, Minbyul and Yoon, Chanwoong},
  year   = {2026}
}

License: Apache-2.0.

Getting the full data

This repository is a 110-row sample. The complete artifacts are not published here:

To license or purchase the full task corpus and environment library — or the complete 2,172,500-task synthesized set — contact wjdalsquf@gmail.com.