KIST-HARILAB/MAVIS

MAVIS (Micro-surgical Artificial Vascular anastomosIS)

This dataset was presented in the paper: SurgMLLMBench: A Multimodal Large Language Model Benchmark Dataset for Surgical Scene Understanding.

<img src="./fig/mavis_overview.png" width="60%" height="60%" style="margin-left: auto; margin-right: auto; display: block;"/>

Dataset Overview

MAVIS is a microsurgical dataset comprising 19 videos of artificial vascular anastomosis procedures performed by three expert microsurgeons at College of Medicine, Korea University, Republic of Korea. For each video frame, it provides:

• —Pixel-level segmentation of seven tool categories
• —Frame-level workflow annotations: surgical stage, phase, and step

This dataset supports research on surgical tool segmentation and surgical workflow recognition in microsurgical environments.

This dataset was developed by the Korea Institute of Science and Technology (KIST) in collaboration with the College of Medicine, Korea University.

<!--

Data Collection

1. 1.Subjects & Cases
2. 2.19 recorded anastomosis sessions on an artificial vessel simulator
3. 3.Surgeon assignments:
4. 4.CASE 01–07: Surgeon 1
5. 5.CASE 08–14: Surgeon 2
6. 6.CASE 15–19: Surgeon 3

1. 1.Acquisition Setup
2. 2.Microscope/Camera model: ???
3. 3.Original resolution: ??? px
4. 4.Frame rate: ??? fps
5. 5.Cropped frame size: 1920 × 1072 px

1. 1.Annotation Tools & Process
2. 2.Segmentation: ???
3. 3.Workflow: Manual tagging of stage/phase/step by the non-medical

1. 1.Annotators
2. 2.Labels applied following a standardized workflow guideline -->

Data Details

Each directory in the MAVIS dataset stores raw image data, segmentation masks, and annotation files.

• —Directory Structure

    MAVIS/
    ├── frames/
    │   ├── CASE01/
    │   │   ├── image_00001.jpg
    │   │   └── ...
    │   ├── ...
    ├── annotations/
    │   ├── long-term.json
    │   ├── short-term.json
    │   ├── segmentations/
    │   │   ├── CASE01/
    │   │   │   ├── image_00001.png
    │   │   │   └── ...
    │   │   ├── ...
    │   └── segmentations_with_keypoint/
    │       ├── CASE01/
    │       │   ├── image_00001.png
    │       │   └── ...
    │       ├── ...
    ├── fig/
    └── README.md

• —Annotation Formats

• —short-term.json: polygon mask data for seven tool classes for each frame
• —long-term.json: stage, phase, and step labels for each frame

• —Stage–Phase–Step Hierarchy

The workflow annotations are structured into six Stages, comprising one or more sequential Phases, which in turn consist of multiple sequential Steps. <center>

</center>

<details> <summary>[Click to expand] Full annotation hierarchy with descriptions</summary>

1. 1.First tying (forming the first knot)
2. 2.Phase: Suturing – place and position the suture
3. 3.Needle holding: grasp the suture needle securely with the needle holder
4. 4.Needle passing: insert the needle through both edges of the vessel and pull it through
5. 5.Needle dropping: release the needle at the optimal position for tying (≈5 o’clock)
6. 6.Phase: Knot tying – create the knot
7. 7.1st knot: wrap the free end of the suture around the instrument and tighten
8. 8.2nd knot: repeat wrapping in the opposite direction and tighten
9. 9.3rd knot: final wrap to secure the stitch
10. 10.Phase: Cutting – trim excess suture
11. 11.Cutting: use scissors to sever both ends of the suture (can cut both at once or sequentially)
12. 12.Second 120° tying (forming the second knot at a position rotated 120° from the first tying)
13. 13.Subtasks: identical to First tying (Suturing → Knot tying → Cutting)
14. 14.Second 180° tying (forming the second knot at a position rotated 180° from the first tying)
15. 15.Subtasks: identical to First tying (Suturing → Knot tying → Cutting)
16. 16.Front side tying (additional knot on the front face between first and second)
17. 17.Subtasks: identical to First tying (Suturing → Knot tying → Cutting)
18. 18.Flip (reorient vessel for back‐side access)
19. 19.Phase: Flip – flip the vessel clamp
20. 20.Flip clamp: reposition the clamp so that the vessel’s backside faces the camera
21. 21.Back side tying (forming knots on the backside between first and second)
22. 22.Subtasks: identical to First tying (Suturing → Knot tying → Cutting)

</details>

<!-- | Class ID | Phase Name | |----------|-------------------| | 0 | Suturing | | 1 | Knot tying | | 2 | Cutting | | 3 | Flip |

-->

• —Tool Segmentation Classes

The dataset contains the following surgical tool classes:

<center>

</center>

Examples of Labeled Data

Figures 1, 2, and 3 show examples of the dataset with segmentation labels.

<center>

</center>

<!-- <figure> <img src="./fig/ex1.png" width="640" height="360"/> <figcaption>Figure 1: Example of Segmentation Mask of Image 1</figcaption> </figure>

<figure> <img src="./fig/ex2.png" width="640" height="360"/> <figcaption>Figure 2: Example of Segmentation Mask of Image 2</figcaption> </figure>

<figure> <img src="./fig/ex3.png" width="640" height="360"/> <figcaption>Figure 3: Example of Segmentation Mask of Image 3</figcaption> </figure> -->

Citation

@misc{choi2025surgmllmbenchmultimodallargelanguage,
      title={SurgMLLMBench: A Multimodal Large Language Model Benchmark Dataset for Surgical Scene Understanding}, 
      author={Tae-Min Choi and Tae Kyeong Jeong and Garam Kim and Jaemin Lee and Yeongyoon Koh and In Cheul Choi and Jae-Ho Chung and Jong Woong Park and Juyoun Park},
      year={2025},
      eprint={2511.21339},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2511.21339}, 
}

Acknowledgment

This work was supported by the Technology Innovation Program (No. RS-2024-00443054) grant funded by the Korea government (the Ministry of Trade, Industry & Energy (MOTIE)).

• —Project Name: Development of a Supermicrosurgical Robot System for Sub-0.8mm Vessel Anastomosis through Human-Robot Autonomous Collaboration in Surgical Workflow Recognition
• —Project Number: RS-2024-00443054