# WebArena

> A realistic web environment benchmark where agents must complete tasks on self-hosted replicas of real websites (Reddit, GitLab, shopping sites, maps, Wikipedia). It evaluates autonomous web navigation and task completion.

## Claim

- **headline**: WebArena success rates support comparing autonomous web agents on end-to-end task completion in self-hosted replicas of real web applications; they do not support claims about the live web or about partial progress.
- **construct**: Autonomous task completion through a browser in realistic web applications
- **population**: 812 natural-language tasks on self-hosted e-commerce, content-management, forum, code-hosting, and map applications
- **measures**: Autonomous web navigation and interaction in realistic website environments; Ability to complete multi-step web tasks like posting, searching, configuring settings, and managing content; Understanding of web UI elements, forms, navigation patterns, and web application workflows; End-to-end task success rate on fully functional, self-hosted web applications
- **doesNotMeasure**: Performance on the open, live web — WebArena uses controlled replicas of fixed websites; Ability to handle dynamic, adversarial, or rapidly changing web content; API or programmatic interaction skills — tasks are designed for browser-based UI interaction; Partial task completion or quality of intermediate steps, as evaluation is binary success/failure

## Instrument

- **itemFormat**: interactive
- **specification**: A task succeeds when the final environment state, or the returned answer for information-seeking tasks, passes the task's programmatic check.
- **anchor**: Task-specific reference checks written by the authors: expected database or page state, URLs, or reference answers
- **evidenceView**: The final state of the web applications and the agent's final answer; intermediate actions are not graded
- **mechanism**: hybrid
- **mechanismDetail**: Tasks that change a site are checked by querying the applications' final state (execution); information-seeking tasks compare the answer with reference strings by exact or must-include matching (deterministic), with an LLM-based fuzzy match for some answers (learned judge).
- **signal**: Binary success/failure per task
- **resolution**: Final state or answer per episode
- **aggregation**: Task success rate over the 812 tasks
- **metrics**: task success rate
- **size**: 812 tasks across 5 web environments
- **methodology**: Deploys self-hosted replicas of five real websites (a Reddit forum, GitLab instance, e-commerce site, map service, and content management system). Agents receive a natural language task instruction and must interact with the websites through browser actions (click, type, navigate) to complete the task. Evaluation uses programmatic checks on the final state of the web environment (e.g., checking if a post was created, a setting was changed) to determine binary success or failure across 812 tasks.

## Regime

- **typicalUses**: measure
- **exposure**: Task intents and evaluation checks are public and the website snapshots are fixed, so agent scaffolds can be tuned to the known task set over repeated runs; there is no hidden split.

## Validation

- **summary**: WebArena's evidence supports reproducible claims about web-agent task completion in realistic settings, because it deploys working web applications and checks the resulting state programmatically. Self-hosted replicas cannot capture the full complexity of the live web — a deliberate trade for reproducibility — and binary success misses partial progress.
- **evidence**: Tasks are achievable, so agent failures reflect the agent; Success checks capture task completion; Results transfer to the live web
- **status**: draft

## Lifecycle

- **saturation**: active
- **contamination**: medium
- **lastReviewed**: 2026-10-12

---
Source: https://benchmaker.ai/benchmarks/webarena