zai-org/Vision2Web

Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification

<div align='center'>

[🏠 Project Page] [📖 arXiv Paper] [🏆 Leaderboard] [📮 Submit Results]

</div>

<p align="center"> <img src="./docs/images/vision2web-cover.png" width="85%"> </p>

Vision2Web is a comprehensive benchmark designed to evaluate multimodal coding agents on visual website development tasks spanning the full software development lifecycle.

This dataset repository contains the benchmark tasks, UI prototypes, test workflows, and resources used to evaluate agent performance.

👀 Introduction

Vision2Web is a hierarchical benchmark for evaluating multimodal coding agents on end-to-end visual website development, measuring their ability to integrate:

• —UI understanding
• —requirements reasoning
• —interactive logic
• —full-stack implementation

in long-horizon development scenarios.

<p align="center"> <img src="./docs/images/compare_bench.png" width="70%"> </p>

The benchmark is organized into three progressive levels:

Level 1 – Static Webpage

Generate responsive executable webpages from multi-device UI prototypes (desktop / tablet / mobile).

Metric

• —Visual Score (VS)

Level 2 – Interactive Frontend

Develop multi-page interactive frontends from multiple prototypes and textual specifications.

Metrics

• —Visual Score (VS)
• —Functional Score (FS)

Level 3 – Full-Stack Website

Build complete full-stack web systems from requirement documents and UI prototypes.

Agents must implement:

• —backend logic
• —state management
• —frontend interactions

Metrics

• —Visual Score (VS)
• —Functional Score (FS)

Evaluation uses a workflow-based agent verification paradigm combining:

• —GUI Agent verifiers for functional correctness
• —VLM-based judges for visual fidelity

This enables scalable and implementation-agnostic evaluation across increasing levels of complexity.

📊 Benchmark Statistics

Vision2Web contains:

• —193 tasks
• —16 subcategories
• —4 major domains

Domains include:

• —E-Commerce
• —SaaS
• —Content Platforms
• —Public Service

The dataset includes:

• —918 prototype images
• —1,255 functional test cases

<table align="center"> <tr> <td align="center" width="50%"> <img src="./docs/images/task_distribution.png" width="100%"/> </td>

<td align="center" width="50%"> <img src="./docs/images/testcasedistribution.png" width="100%"/><br/><br/> <img src="./docs/images/compare_task.png" width="80%"/> </td> </tr> </table>

📥 Using the Dataset

The dataset can be downloaded directly from Hugging Face.

After downloading, extract the dataset and place it in your project directory with the following structure:

datasets/
├── webpage/    # Level 1: Static Webpage (100 tasks)
├── frontend/   # Level 2: Interactive Frontend (66 tasks)
└── website/    # Level 3: Full-Stack Website (27 tasks)

Each task directory contains the following components:

Once extracted, ensure the dataset directory is placed at the root of the Vision2Web project so that the evaluation pipeline can locate the benchmark tasks correctly.

⚠️ License

Vision2Web is released under the CC-BY-NC-SA-4.0 license.

✒️ Citation

If you find Vision2Web useful in your research, please cite:

@misc{he2026vision2webhierarchicalbenchmarkvisual,
      title={Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification},
      author={Zehai He and Wenyi Hong and Zhen Yang and Ziyang Pan and Mingdao Liu and Xiaotao Gu and Jie Tang},
      year={2026},
      eprint={2603.26648},
      archivePrefix={arXiv},
      primaryClass={cs.SE},
      url={https://arxiv.org/abs/2603.26648},
}