Datasetpublic

martincousseau/mermaid-persona-queries

Mermaid Persona Queries (v1) 50 hand-written user queries for evaluating and benchmarking text-to-Mermaid diagram generation, authored across 5 distinct user personas. Users exist within a single company spanning many roles (engineering, HR, finance, ops, support, product, legal, IT) — the dataset varies how people ask, not a business domain. Motivation Real users of text-to-diagram tools do not write uniform prompts. Some paste truncated meeting notes, some… See the full description on the dataset page: https://huggingface.co/datasets/martincousseau/mermaid-persona-queries.

sourceHugging Facecc-by-4.0updated 7d agoView on Hugging Face

0likes41downloads

Dataset Card

Mermaid Persona Queries (v1)

50 hand-written user queries for evaluating and benchmarking text-to-Mermaid diagram generation, authored across 5 distinct user personas. Users exist within a single company spanning many roles (engineering, HR, finance, ops, support, product, legal, IT) — the dataset varies how people ask, not a business domain.

Motivation

Real users of text-to-diagram tools do not write uniform prompts. Some paste truncated meeting notes, some dictate exact Mermaid specs, some describe graphs formally, and some think out loud with self-corrections mid-sentence. Models evaluated only on clean prompts overestimate their robustness. This dataset provides a small, controlled stress set for persona-conditioned evaluation.

The 5 Personas

ID	Name	Style	Description
P1	Maya	`thinking_out_loud`	Messy stream-of-consciousness drafter. Filler words, self-corrections ("wait, no"), retracted branches, intent buried at the end.
P2	David	`precise_architect`	Knows Mermaid. Names the diagram type, gives exact nodes/edges/labels, expects literal compliance with no invented extras.
P3	Priya	`mathematician`	Formal notation: sets, relations, transition functions, probability matrices, cardinalities. Expects faithful translation of the formalism.
P4	Sam	`everyday_user`	Plain, polite, complete one-or-two-sentence requests. No Mermaid knowledge; expects sensible defaults.
P5	Leo	`copy_paster_noisy`	Pastes raw material (meeting notes, logs, emails, CSV exports), often truncated mid-word, with vague instructions ("flowchart this pls??").

Full persona definitions: `personas.md` and the personas config.

Data Fields

`queries` config (50 rows):

Field	Type	Description
`id`	string	Query ID, `Q01`–`Q50`. Q01–Q10 = P1, Q11–Q20 = P2, Q21–Q30 = P3, Q31–Q40 = P4, Q41–Q50 = P5.
`query`	string	The raw user message, verbatim, including typos, noise, and truncations.
`persona_id`	string	`P1`–`P5`.
`persona_name`	string	Persona display name.
`persona_style`	string	Machine-readable style label.

`personas` config (5 rows): persona_id, persona_name, persona_style, description.

Usage

python

from datasets import load_dataset

ds = load_dataset("martincousseau/mermaid-persona-queries")          # queries (default)
personas = load_dataset("martincousseau/mermaid-persona-queries", "personas")

# Filter by persona
leo_queries = [r for r in ds["train"] if r["persona_id"] == "P5"]

Coverage

—Diagram types exercised: flowchart, sequence diagram, state diagram (v1/v2), ER diagram, Gantt chart, org chart, mind map, class diagram, git graph, timeline, bipartite/directed graphs.
—Input conditions: clean prompts, self-corrected prompts, formal specifications, truncated pastes (mid-word cutoffs preserved), embedded log lines, CSV exports, email thread junk, broken wiki formatting.
—Company roles referenced: hiring (HR), support tiers, DevOps/CI, finance approvals, product launches, office management, sales, data engineering.

This is an input-only dataset: no reference Mermaid outputs are included. It is intended for generation evaluation (LLM-as-judge, rubric scoring, or pairwise comparison), not supervised fine-tuning.

Source & Curation

Queries were hand-authored for this dataset (v1). No personal data, no real company data; all names, systems, and scenarios are fictional. Truncations and noise in P5 queries are deliberate — do not "fix" them.

Limitations

—50 items: suitable for qualitative eval and regression test suites, not for statistically significant leaderboard claims.
—English only; personas reflect English-language office communication styles.
—Persona labels are author-assigned, not derived from real user studies.

License

CC-BY-4.0. Attribution appreciated.

Citation

bibtex

@misc{mermaid-persona-queries,
  title  = {Mermaid Persona Queries: 50 persona-conditioned prompts for text-to-diagram evaluation},
  year   = {2026},
  note   = {v1},
  howpublished = {\url{https://huggingface.co/datasets/martincousseau/mermaid-persona-queries}}
}