Datasetpublic

pritamdeka/BPMN-VLM

🏗️ BPMN Diagram → BPMN XML Paired Dataset Structured Extraction from Business Process Diagrams using Vision-Language Models This dataset contains Business Process Model and Notation (BPMN) diagrams paired with their corresponding .bpmn XML ground truth files.The dataset is designed for training, evaluation, and benchmarking multimodal models that perform structured extraction from diagrams, including OCR-enhanced pipelines and vision-language models (VLMs).… See the full description on the dataset page: https://huggingface.co/datasets/pritamdeka/BPMN-VLM.

sourceHugging Faceupdated 10mo agoView on Hugging Face

1likes56downloads

Dataset Card

🏗️ BPMN Diagram → BPMN XML Paired Dataset

Structured Extraction from Business Process Diagrams using Vision-Language Models

This dataset contains Business Process Model and Notation (BPMN) diagrams paired with their corresponding .bpmn XML ground truth files. The dataset is designed for training, evaluation, and benchmarking multimodal models that perform structured extraction from diagrams, including OCR-enhanced pipelines and vision-language models (VLMs).

📦 Dataset Contents

Each example includes:

Field	Description
`image`	BPMN diagram image (PNG/JPEG), uploaded directly to HF
`bpmn`	Text content of the corresponding `.bpmn` XML file
`image_filename`	Original image filename
`bpmn_filename`	Original BPMN filename
`split`	One of: `train`, `validation`, `test`

Folder structure used during creation:

dataset/
├── train/
│   ├── images/
│   ├── bpmn/
├── validation/
│   ├── images/
│   ├── bpmn/
├── test/
    ├── images/
    ├── bpmn/

🖼️ Example Image

Example BPMN Image

Ground Truth BPMN (excerpt)

<?xml version='1.0' encoding='UTF-8'?>

<bpmn:definitions xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:bpmn="http://www.omg.org/spec/BPMN/20100524/MODEL"
    xmlns:bpmndi="http://www.omg.org/spec/BPMN/20100524/DI"
    xmlns:dc="http://www.omg.org/spec/DD/20100524/DC"
    id="Definitions_1"
    targetNamespace="http://bpmn.io/schema/bpmn">

  <bpmn:process id="Process_1" isExecutable="false">

    <bpmn:startEvent id="StartEvent_1">
      <bpmn:outgoing>Flow_0abcd12</bpmn:outgoing>
    </bpmn:startEvent>

    <bpmn:task id="Task_1" name="Receive Order">
      <bpmn:incoming>Flow_0abcd12</bpmn:incoming>
      <bpmn:outgoing>Flow_0efgh34</bpmn:outgoing>
    </bpmn:task>

    <bpmn:exclusiveGateway id="Gateway_1">
      <bpmn:incoming>Flow_0efgh34</bpmn:incoming>
      <bpmn:outgoing>Flow_0ijkl56</bpmn:outgoing>
      <bpmn:outgoing>Flow_0mnop78</bpmn:outgoing>
    </bpmn:exclusiveGateway>

    <bpmn:task id="Task_2" name="Validate Order">
      <bpmn:incoming>Flow_0ijkl56</bpmn:incoming>
      <bpmn:outgoing>Flow_0qrst90</bpmn:outgoing>
    </bpmn:task>

      ...

 </bpmndi:BPMNDiagram>
</bpmn:definitions>

🔧 Usage

python

from datasets import load_dataset

ds = load_dataset("pritamdeka/BPMN-VLM")

example = ds["train"][0]

image = example["image"]               # PIL image object
bpmn_text = example["bpmn"]           # XML content as string
image_name = example["image_filename"]
bpmn_name = example["bpmn_filename"]

🎯 Applications

This dataset is suitable for:

—BPMN diagram understanding and parsing
—OCR + VLM multimodal pipelines
—Structured JSON extraction
—Diagram-to-XML reconstruction
—Fine-tuning Pixtral, Qwen2.5-VL, LLaMA 3.2 Vision, Aya Vision, Gemma3 and other VLMs
—Evaluation against ground truth .bpmn files

Ideal for research in:

—Vision-language reasoning
—Diagram understanding
—Business process modelling automation

📜 Citation

If you use this dataset, please cite:

bibtex

@misc{deka2025structuredextractionbusinessprocess,
      title={Structured Extraction from Business Process Diagrams Using Vision-Language Models}, 
      author={Pritam Deka and Barry Devereux},
      year={2025},
      eprint={2511.22448},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2511.22448}, 
}

📄 License

This dataset is released under CC BY-NC 4.0 — It can be used for research and non-commercial purposes with attribution.

🙏 Acknowledgements

Developed at the Advanced Research Centre (ARC), Queen’s University Belfast, as part of research into multimodal structured extraction from business process diagrams.

📬 Contact

For questions, contact: Pritam Deka — p.deka@qub.ac.uk