TIGER-AI-Lab/WebsiteBench
WEBSITE RECONSTRUCTION BENCHMARK FOR AGENTS
Benchmark for evaluating the ability of AI agents to reconstruct full-featured websites through browser exploration. Contains 70 tasks with 46,916 hidden test cases for verifying visual and functional fidelity.
// KEY FEATURES
- 70 production websites and 46,916 hidden test cases
- Evaluation by SSIM 0.99 and functional checks
- Execution via Harbor with Docker isolation
- Support for Codex, Oracle and other agents
// DEEP DIVE: HOW IT WORKS
WebsiteBench tasks agents with studying a live website in a browser and reproducing its full-stack implementation. Evaluation uses hidden test cases (302–2016 per task): a case passes when functional checks succeed and visual checkpoints reach normalized SSIM ≥ 0.99. Task reward is the share of passing cases.
The suite includes 70 websites of varying complexity, hosted on Harbor Hub and Hugging Face. Infrastructure runs on Harbor: isolated Docker containers with access to external registries (npm, GitHub, Debian). Multiple agents are supported — Codex, Oracle (reference solution) and others via configuration. Tasks are downloaded via the websitebench PyPI package.
Requires Docker, Harbor CLI (uv tool install harbor==0.25.0) and OPENAI_API_KEY. Running all 70 tasks uses a single harbor run command with network host allowlist flags. Reference sites are for non-commercial research; benchmark code is Apache-2.0. A canary GUID is included for exclusion from training data.
// USE CASES
Best for: AI agent researchers and browser automation developers
- Evaluate agent quality of layout and logic reconstruction
- Compare different LLMs on website reverse-engineering tasks
- Test agent robustness on 70 real production websites
// QUICK START
pip install websitebench
websitebench download --task medium
uv tool install harbor==0.25.0Commands are taken from the repository README — check the current version on GitHub before running.
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram