zai-org/Vision2Web
Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification [🏠 Project Page] [📖 arXiv Paper] [🏆 Leaderboard] [📮 Submit Results] Vision2Web is a comprehensive benchmark designed to evaluate multimodal coding agents on visual website development tasks spanning the full software development lifecycle. This dataset repository contains the benchmark tasks, UI prototypes, test workflows, and resources used to evaluate agent… See the full description on the dataset page: https://huggingface.co/datasets/zai-org/Vision2Web.
231.4k
1---2license: apache-2.03task_categories:4- text-generation5language:6- en7tags:8- agent9size_categories:10- n<1K11configs:12- config_name: webpage13 data_files:14 - split: test15 path: "webpage/test.parquet"16 17- config_name: frontend18 data_files:19 - split: test20 path: "frontend/test.parquet"21 22- config_name: website23 data_files:24 - split: test25 path: "website/test.parquet"26---27 28# Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification29 30313233 34<div align='center'>35 36[[🏠 Project Page](https://vision2web-bench.github.io/)] [[📖 arXiv Paper](https://arxiv.org/abs/2603.26648)] [[🏆 Leaderboard](https://vision2web-bench.github.io/#leaderboard)] [[📮 Submit Results](https://huggingface.co/datasets/zai-org/Vision2Web-Leaderboard)]37 38</div>39 40<p align="center">41 <img src="./docs/images/vision2web-cover.png" width="85%">42</p>43 44Vision2Web is a comprehensive benchmark designed to evaluate multimodal coding agents on **visual website development tasks spanning the full software development lifecycle**.45 46This dataset repository contains the **benchmark tasks, UI prototypes, test workflows, and resources** used to evaluate agent performance.47 48---49 50# 👀 Introduction51 52Vision2Web is a hierarchical benchmark for evaluating multimodal coding agents on **end-to-end visual website development**, measuring their ability to integrate:53 54- UI understanding55- requirements reasoning56- interactive logic57- full-stack implementation58 59in **long-horizon development scenarios**.60 61<p align="center">62 <img src="./docs/images/compare_bench.png" width="70%">63</p>64 65The benchmark is organized into three progressive levels:66 67### Level 1 – Static Webpage68 69Generate responsive executable webpages from multi-device UI prototypes 70(desktop / tablet / mobile).71 72**Metric**73 74- Visual Score (VS)75 76---77 78### Level 2 – Interactive Frontend79 80Develop multi-page interactive frontends from multiple prototypes and textual specifications.81 82**Metrics**83 84- Visual Score (VS)85- Functional Score (FS)86 87---88 89### Level 3 – Full-Stack Website90 91Build complete full-stack web systems from requirement documents and UI prototypes.92 93Agents must implement:94 95- backend logic96- state management97- frontend interactions98 99**Metrics**100 101- Visual Score (VS)102- Functional Score (FS)103 104---105 106Evaluation uses a **workflow-based agent verification paradigm** combining:107 108- **GUI Agent verifiers** for functional correctness109- **VLM-based judges** for visual fidelity110 111This enables **scalable and implementation-agnostic evaluation** across increasing levels of complexity.112 113---114 115# 📊 Benchmark Statistics116 117Vision2Web contains:118 119- **193 tasks**120- **16 subcategories**121- **4 major domains**122 123Domains include:124 125- E-Commerce126- SaaS127- Content Platforms128- Public Service129 130The dataset includes:131 132- **918 prototype images**133- **1,255 functional test cases**134 135<table align="center">136<tr>137<td align="center" width="50%">138 <img src="./docs/images/task_distribution.png" width="100%"/>139</td>140 141<td align="center" width="50%">142 <img src="./docs/images/test_case_distribution.png" width="100%"/><br/><br/>143 <img src="./docs/images/compare_task.png" width="80%"/>144</td>145</tr>146</table>147 148---149 150# 📥 Using the Dataset151 152The dataset can be downloaded directly from Hugging Face.153 154After downloading, extract the dataset and place it in your project directory with the following structure:155 156```157 158datasets/159├── webpage/ # Level 1: Static Webpage (100 tasks)160├── frontend/ # Level 2: Interactive Frontend (66 tasks)161└── website/ # Level 3: Full-Stack Website (27 tasks)162 163```164 165Each task directory contains the following components:166 167| File / Folder | Description |168|---|---|169| `prototypes/` | UI prototype images (desktop / tablet / mobile) |170| `resources/` | Multimedia assets used in tasks |171| `workflow.json` | Functional test workflow specification |172| `prompt.txt` | Textual requirements (Level 2 only) |173| `prd.md` | Requirement document (Level 3 only) |174 175Once extracted, ensure the dataset directory is placed at the root of the Vision2Web project so that the evaluation pipeline can locate the benchmark tasks correctly.176 177---178 179# ⚠️ License180 181Vision2Web is released under the **CC-BY-NC-SA-4.0 license**.182 183---184 185# ✒️ Citation186 187If you find Vision2Web useful in your research, please cite:188 189```bibtex190@misc{he2026vision2webhierarchicalbenchmarkvisual,191 title={Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification},192 author={Zehai He and Wenyi Hong and Zhen Yang and Ziyang Pan and Mingdao Liu and Xiaotao Gu and Jie Tang},193 year={2026},194 eprint={2603.26648},195 archivePrefix={arXiv},196 primaryClass={cs.SE},197 url={https://arxiv.org/abs/2603.26648},198}199```