xiaoooobai/SVGenius

SVGenius: Benchmarking LLMs in SVG Understanding, Editing and Generation

<!-- Provide a quick summary of the dataset. --> <a href="https://arxiv.org/abs/2506.03139" target="blank"> <img alt="arXiv" src="https://img.shields.io/badge/arXiv-SVGenius-red?logo=arxiv" height="20" /> </a> <a href="https://huggingface.co/datasets/xiaoooobai/SVGenius" target="blank"> <img alt="SVGenius" src="https://img.shields.io/badge/%F0%9F%A4%97%20Benchmark-SVGenius-ffc107?color=ffc107&logoColor=white" height="20" /> </a> <a href="https://zju-real.github.io/SVGenius/" target="blank"> <img alt="Webpage" src="https://img.shields.io/badge/%F0%9F%8C%8E_Website-SVGenius-green.svg" height="20" /> </a>

We introduce SVGenius, the first large-scale complexity-stratified benchmark accessing (M)LLMs capabilities across three progressive dimensions: Understanding (perceptua and semantic QA), Editing (bug fixing, code optimization, style editing), and Generation (text-to-SVG, image-to-SVG, style transfer). Built on real-world data from 24 application domains with systematic complexity stratification, SVGenius evaluates models through 8 task categories and 18 metrics. We assess 22 mainstream models spanning different scales, architectures, training paradigms, and accessibility levels. <img alt="SVGenius Dataset Overview" src="https://cdn.jsdelivr.net/gh/ZJU-REAL/SVGenius@main/docs/static/images/overview.jpg" style="width: 100%; max-width: 1000px;" />

The dataset contains the following fields: | Field Name | Description | | :--------- | :---------- | | id | Unique identifier for each SVG icon sample | | filename | Original filename of the SVG file, preserving the source naming convention with category and identifier information | | difficulty | Complexity level of the SVG icon, includes 3 distinct categories: easy, medium, and hard for evaluating different levels of SVG processing capabilities | | svg_code | Complete SVG markup code containing the vector graphics definition, including all paths,styles, and attributes |

• —Language(s) (NLP): en, zh
• —License: mit

Uses

from datasets import load_dataset
ds = load_dataset("xiaoooobai/SVGenius")

Citation

@misc{chen2025svgeniusbenchmarkingllmssvg,
      title={SVGenius: Benchmarking LLMs in SVG Understanding, Editing and Generation}, 
      author={Siqi Chen and Xinyu Dong and Haolei Xu and Xingyu Wu and Fei Tang and Hang Zhang and Yuchen Yan and Linjuan Wu and Wenqi Zhang and Guiyang Hou and Yongliang Shen and          Weiming Lu and Yueting Zhuang},
      year={2025},
      eprint={2506.03139},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2506.03139}, 
}