VisionWebDev: A Hierarchical Benchmark for Visual Website Development with Agent Verification
What is VisionWebDev
VisionWebDev is a benchmark for evaluating whether multimodal coding agents can build real websites from visual prototypes and structured requirements. It goes beyond small code edits and static UI generation to measure end-to-end web development ability in realistic settings.
Each task provides multimodal inputs such as UI prototype images, requirement descriptions, and development assets. Agents are expected to generate executable websites that satisfy both functional behavior and visual fidelity.
To support reliable evaluation, VisionWebDev introduces an automated verification framework that combines workflow-driven GUI testing with VLM-based visual judging.
Why VisionWebDev
Existing coding benchmarks mainly focus on localized code edits, while most multimodal website benchmarks are limited to static webpage reproduction. These settings do not fully capture the complexity of modern web development, where agents must reason over visual layouts, interaction flows, application state, and system behavior across multiple pages.
VisionWebDev closes this gap by evaluating the full spectrum of visual website development, from responsive UI implementation to interactive frontend engineering and complete full-stack applications.
Benchmark Highlights
193
Tasks
918
Prototype Images
1,255
Test Cases
16
Categories
VisionWebDev spans 16 subcategories across 4 major domains and covers progressively harder development settings, from static responsive webpages to interaction-heavy frontends and requirement-driven full-stack systems.