1. Introduction: the web won’t hold still

If you have built a web agent recently, you know the failure pattern. You give it a task like “pull every listing from this directory into a spreadsheet” and watch it inch forward. It reads the page. It predicts a click. It waits for the new DOM, the page structure the browser sees. It reads again, predicts again, waits again.

Then, somewhere around step 40, things fall apart. A modal pops up unexpectedly. The “next page” button moves. The agent mistakes one element for another. Any one of these can derail the whole task. The deeper problem isn’t the bad click. It’s how the agent operates: look at the page, decide on one action, see what changed, then decide again. It repeats this loop over and over, without a durable plan for how to complete the task from start to finish.

The field has tried a few different ways to make this loop more reliable. Some agents, like OpenAI’s Operator and Anthropic’s Computer Use, work from screenshots and interact with a website much like a person would. Others, like WebVoyager, use the page’s DOM to understand what elements are available and decide which one to interact with.

Benchmarks such as Mind2Web and WebArena made these systems easier to compare by giving agents a standard set of actions—click, type, scroll, select. And open-source tools like browser-use, Skyvern, Stagehand, and LaVague have packaged these ideas into APIs that engineers can more easily build into real applications.

These approaches make the loop more reliable, but they don’t change how it fundamentally works: the agent still takes one action at a time, waits to see what happens, then decides what to do next. And when the task is over, it hasn’t built anything reusable—it has only completed a sequence of clicks.

Webwright, a browser-agent framework from Microsoft Research and the University of Hong Kong, takes a different approach. Its tagline captures the idea: “A terminal is all you need for web agents.”

Instead of asking the model to figure out the next click, Webwright has agents writing and running code—using bash and Playwright scripts to open browsers, inspect pages, and carry out the task. The result isn’t just a long sequence of browser actions. It’s a program engineers can inspect, rerun, modify, and reuse.

This difference matters most when the web is your data source: dashboards, product catalogs, search results, internal tools, JavaScript-heavy sites, and workflows you expect to run more than once. In those cases, the question isn’t just whether an agent can finish the task. It’s whether it should keep clicking through the browser or write a reusable program to do the work.

We’ll start with the four main approaches to building web agents and the limitations they still share. Then we’ll look inside Webwright: how its three core components work, how a framework of roughly a thousand lines performs on benchmarks, and what the results say about cost and reliability. Finally, we’ll put the approach to work on three common scraping problems—paginated pages, JavaScript-rendered content, and infinite-scroll feeds.

2. Why web agents keep breaking

The shift to “write code” matters because it addresses the underlying problem, not just the symptoms. Today’s web agents differ in how they understand a page—some look at screenshots, others read the DOM—but most still work the same way: take one browser action, see what happens, then decide on the next one.

That works for short tasks. But the longer the task runs, the more chances there are for one bad click, a changed page, or a misread element to throw everything off.

| Family | What the model sees | Why it helps | Where it breaks |
|---|---|---|---|
| Vision agents | Screenshots | Works when the page is only visually understandable | Layout shifts, pixel ambiguity, expensive screenshots |
| DOM / set-of-marks agents | HTML, accessibility trees, numbered boxes | More grounded than raw pixels | Huge page state, changing element IDs, hard grounding |
| Fixed action-API agents | A menu like click/type/scroll/select | Reproducible benchmark loop | Cannot express loops, retries, file output, or “do this for every row” |
| Browser frameworks | Packaged browser-control loops | Easier to ship and observe | Often still per-step, session-centered, and artifact-poor |

Vision agents are easy to understand: they look at the page much like a person does and decide where to click. That works well for many browser tasks, but scraping demands more consistency. A small layout shift can move a button just enough for the agent to click the wrong place—or nothing at all.

There’s also a cost to repeatedly looking at screenshots. Every new screenshot consumes tokens, and the agent has to carry enough context forward to remember what it already did. On a long task, that context becomes harder and more expensive to maintain.

DOM and accessibility-tree agents avoid some of the problems that come with screenshots. Instead of guessing where an element is based on pixels, they can read the structure of the page and identify buttons, links, forms, and other elements directly. WebVoyager, for example, reported about 59% task success across 50 real-world websites, significantly better than text-only baselines.

But this creates a different problem: too much page data. A complex page can produce an accessibility tree larger than 50KB. As the agent moves through a task, old page state accumulates in its context even though much of it is no longer useful. The references it uses to identify elements can also change when a page lazy-loads content, rerenders, or navigates somewhere new.

So better access to page structure doesn’t necessarily make long browser tasks reliable. On VisualWebArena, leading vision-language agents completed only about 16% of tasks, compared with roughly 89% for humans.

Fixed action APIs made web agents easier to build and benchmark. Give the model a small set of actions—click, type, scroll, select—then let it observe the result and choose again. The downside is that the agent can only express one small step at a time. It can’t naturally say, “keep clicking next until there are no pages left,” “retry if this element doesn’t appear,” or “collect these 1,000 rows and save them to a CSV.” Each of those has to be broken into many individual actions, with another model call in between.

Frameworks like browser-use, Skyvern, Stagehand, and LaVague make browser agents much easier to build and integrate. That’s useful, but it doesn’t solve one important problem for recurring data work: when the task is finished, there often isn’t anything reusable left behind. The agent may have collected the data once, but next week it has to work through the browser all over again.

Research had already pointed toward another approach. In the ICML 2024 paper Executable Code Actions Elicit Better LLM Agents, the researchers behind CodeAct replaced predefined, JSON-style actions with executable Python. They reported up to 20% higher success rates while using roughly 30% fewer steps.

The reason is straightforward: code lets the model do more than take one action at a time. It can use loops, store variables, retry failures, write files, and inspect errors—all within a program it can run again. Webwright brings that same idea to browser automation.

3. Webwright’s unique point: make the workspace the state

Most browser agents keep their progress in the browser session. Close the tab, and that state is gone.

Webwright flips this around. The browser is temporary; the local workspace is what persists. The agent writes scripts, logs, screenshots, and output files as it works, eventually turning a successful run into a reusable tool. This shift has a few practical benefits.

  • More robust interactions— Playwright selectors and wait conditions are more reliable than pixel coordinates or temporary element IDs.
  • Better composition— Loops and functions can handle hundreds of repeated actions in one program
  • Visible state— Progress is visible in files and logs.
  • Reusable output— Once the task works, the code can be reproduced instead of starting from scratch.

Where the project’s four stated advantages come from:

  • Robust, reusable interactions— the agent acts through queries and wait-for-condition checks (- page.locator(...),- wait_for_selector(...)) instead of pixel coordinates or frozen element IDs, so a script survives layout shifts and re-renders.
  • Efficient composition— loops, functions, and variables let a single turn say “do this for every row,” work a one-action-at-a-time agent has to spell out step by step.
  • Workspace as state— progress lives in files, not in a fragile session or a context window bloated with stale page dumps.
  • Minimal by design— the whole system leans on four libraries (- httpx,- pydantic,- playwright,- typer) with no hidden framework beneath, and still posts state-of-the-art numbers.

3.1 Let’s do quick comparisons between Webwright and other options in some scenarios.

3.1.1 Demo 1 · When clicking isn’t precise enough

The task: Use Chase’s IRA calculator to compare a Traditional vs. Roth IRA for someone who is 30, retires at 65, saves $300 a month, earns a 3% return, and has tax rates of 13% today and 24% in retirement.

The challenge is that the calculator uses six interactive JavaScript sliders.

Result:

  • Webwrightsets the values directly in code by updating the DOM inputs and triggering the required events. The values are exact, the chart renders correctly, and the working solution is saved as a reusable script.

  • A vision agenthas to manipulate the sliders visually. It gets close, but not close enough: the $300 contribution lands at $294.

3.1.2 Demo 2 · When the same task comes back

The task: Search Google Flights for a round trip from Seattle to San Francisco, including the dates, and return the ranked results.

Result:

  • Webwright completes the search like other browser agents might. The important difference is what happens afterward: it keeps the working code.

  • When a similar flight search comes up later, the agent doesn’t have to figure out every field, date picker, and click again. It can reuse the previous script, change the inputs, and run it again.

That’s what Microsoft means by “your browsing history is code instead of clicks.” A completed task becomes a starting point for the next one, rather than a browser session that disappears when it ends.

3.2 How Webwright differs from other browser-agent repos

The alternatives are useful, but they still put the browser at the center of the workflow. Stagehand combines Playwright with natural-language commands. agent-browser gives agents a CLI for taking small browser actions. browser-use repeatedly reads the page, chooses an action, and executes it.

Webwright takes a different approach. Instead of choosing the next browser action, the model can write an entire Python script. The browser is temporary; the code, logs, and outputs stay in the local workspace. And when the task is solved, the agent leaves behind a program that can be run again.

That’s the core idea: clicking completes the task once; code completes it and keeps the solution.

3.3 Inside Webwright

So what does it take to build an agent like this? Surprisingly little.

Most web agents put a harness—the software connecting the model to the browser—between the two. That harness usually gives the model a fixed set of actions: click this element, type into this field, scroll the page, read the DOM, take a screenshot.

Webwright takes a different approach. Instead of giving the model a menu of browser actions, it gives the model a terminal and lets it decide what commands to run.

That makes the system surprisingly small. The core harness is roughly 1,000 lines of code across three components. The full repository is closer to 1,500 lines once you include the command-line interface and support for different model providers. There is no large library of predefined browser actions. No custom DOM engine. The core system is just three pieces:

  • Runner(~150 lines) — Keeps track of the task and everything that has happened so far: what the agent is trying to do, the current state of its workspace, and the results of previous actions.
  • Model Endpoint(~550 lines) — Connects Webwright to the language model. It provides backends for OpenAI, Anthropic, and OpenRouter.
  • Environment(~300 lines) — Gives the model a terminal connected to Playwright running Chromium. This is where commands actually execute, browser interactions happen, and files created during the task are stored.

The interaction between these pieces is a simple loop.

  • The Runner gives the model the task and the latest context.
  • The model decides what to do next and returns a shell command.
  • The Environment runs that command and sends back what happened—terminal output, logs, screenshots, or error messages.
  • Webwright adds those results to the context and asks the model what to do next.

In short, the loop looks like this:

understand the current state → choose a command → run it → see what happened → repeat

The process continues until the model believes the task is complete and a final self-check agrees. Webwright does not try to encode every possible browser interaction into the harness. It gives the model a general-purpose interface—the terminal—and lets the model figure out how to use it.

The benchmarks support the design. On Online-Mind2Web, GPT-5.4 with Webwright scores 86.7%, the highest among open-source AutoEval harnesses, while Claude Opus 4.7 reaches 84.7% and performs better on the hardest tasks.

The bigger signal comes from Odysseys. GPT-5.4 using coordinate-based browser control scores 33.5%. With Webwright, the same model reaches 60.1%—a 26.6-point gain from changing the harness, not the model.

Webwright’s project page lists 60.8%; I use the 60.1% reported in its GitHub comparison for consistency.

Another result supports the thesis: once Webwright has built reusable tools, the model can get smaller. Microsoft reports that even a 9B open model (Qwen-3.5-9B) performs well on Online-Mind2Web once five or more tools are available. The tool does not just save work—it lowers the model capability needed next time.

There are tradeoffs. These are LLM-judged AutoEval scores, and the headline Mind2Web result uses 100 of 300 tasks. It is also not cheap: about $2.37 per task with GPT-5.4 and $6.09 with Claude Opus 4.7. Webwright spends more compute upfront to build tools that are more robust and reusable.

4. Experiments and Result

I tested Webwright on three increasingly difficult sites, using a separate Claude Sonnet agent for each run. This is to see how robust Webwright is where scraping usually breaks.

I used the Claude Code plugin rather than the standalone benchmark harness. It keeps the same core setup—terminal + Playwright—but Claude Code runs the agent loop. That removes the need for a separate API key or per-task API bill, though not the compute cost.

The tradeoff is token usage. In Microsoft’s example, the Codex-hosted skill used ~3.3M tokens versus ~424K for the standalone harness—about 8× more, largely from cached context. The cost shifts into the host session rather than disappearing.

Setup took one command:

playwright install firefox # the Claude Code skill drives headless Firefox, ~110MB one-time### 4.1 🔧 Test 1 — static pagination · books.toscrape.com

books.toscrape.com is the easiest one among 3 cases: 50 numbered catalogue pages, 20 books each, served as plain HTML. The task was to extract every book title, price, rating, availability, and URL, then craft a reusable CLI with --pages and --out.

Before writing the scraper, the agent inspected the site and tested its boundaries: page 50 had no next link, while page 51 returned a 404. It then pulled selectors from a real product card and built a simple pagination loop.

for n in range(1, pages + 1): url = CATALOGUE_URL_TEMPLATE.format(n=n) await page.goto(url, wait_until="domcontentloaded") cards = page.locator("article.product_pod") count = await cards.count() log(n, f"loaded catalogue page {n}/{pages} ({url}) -> {count} book cards found")
A click-based agent could handle this site, but code was cleaner. One subtle issue was relative book URLs, which change across pages. Instead of manually constructing them, the scraper used the browser-resolved href values.

The result was not just scraped data, but a standalone CLI tool that could run again without an agent.

$ python final_script.py --help usage: final_script.py [-h] [--pages PAGES] [--out OUT] Scrape all books listed on books.toscrape.com's catalogue pages. --pages PAGES Number of catalogue pages to traverse ... Default: 50. --out OUT Output CSV file path ... Default: books.csv. $ python final_script.py --pages 2 --out sample.csv -> 40 rows written to .../sample.csv
Result: 1,000 books across 50 pages, zero empty fields, in ~37 seconds.

The verification step also caught a bug: the first version accidentally cleared its evidence log when running --help. The agent found the side effect, fixed it, tested the fix, and reran successfully. Even on this simple site, the advantage was clear: a debuggable, reusable program instead of a one-time click trace.

4.2 🔧 Test 2 — JavaScript-rendered · quotes.toscrape.com/js

The second test adds JavaScript. The quotes are not present in the raw HTML; they appear only after the browser runs the page’s JavaScript. The agent verified this first: a direct HTTP request returned zero quote elements, while the rendered page showed 10. A basic requests + BeautifulSoup scraper would silently return nothing.

That means every page must be rendered before extraction. There is another catch: page 11 still returns HTTP 200, so status codes cannot tell the scraper when to stop. Instead, the program checks the live DOM for the next link and stops when it disappears on page 10.

while True: url = BASE_URL if n == 1 else PAGE_URL_TEMPLATE.format(n=n) await page.goto(url, wait_until="domcontentloaded") await page.wait_for_selector(".quote", timeout=10000) # wait for JS to inject the quotes ... # read the 10 rendered .quote cards has_next = await page.locator("li.next a").count() > 0 if not has_next or n >= pages: # stop on the DOM, not a status code break n += 1
Again, the output became a reusable CLI with --pages and --out, able to run without an agent.

The payoff — the crafted CLI. As on the books run, the working script became a reusable scrape_quotes(pages, out) tool with an argparse interface (--pages, default 10; --out, default quotes.csv) that re-runs standalone, no agent in the loop.

Result: 100 quotes across 10 pages in 8.9 seconds, with zero empty text or author fields.

The run also exposed a bad assumption in my brief: I expected two pages to produce 40 rows, borrowing the 20-per-page count from the books site. This site serves 10, so the correct result was 20. The agent returned the real data and flagged the mismatch rather than forcing the output to fit the spec.

4.3 🔧 Test 3 — infinite scroll · quotes.toscrape.com/scroll

The third test removes pagination entirely. Quotes load 10 at a time via AJAX as the page scrolls, so there are no page URLs to iterate through. The scraper has to scroll, wait for new content, measure the page, and decide when loading is finished.

The agent first confirmed the site’s behavior: has_next becomes false at page 10, and page 11 returns no quotes. It then built a scroll-until-stable loop that stops when no new content appears.

for i in range(1, max_scrolls + 1): await page.evaluate("window.scrollTo(0, document.body.scrollHeight)") await page.wait_for_timeout(1000) count = await page.locator(".quote").count() if count == prev_count: # no new quotes arrived stable_iters += 1 if stable_iters >= 2: # stop on stability, not a fixed count break else: stable_iters = 0 prev_count = count
The DOM count grew 10 -> 20 -> ... -> 100, held at 100 for two scrolls, and stopped at iteration 11. --max-scrolls was only a safety backstop. Forcing --max-scrolls 5 returned exactly 60 rows, proving the stop condition answered to the page, not a hidden constant.

Result: 100 quotes in 11 iterations, taking ~17 seconds.

Across the three tests, the value of code becomes clearer as the sites get harder. Static pagination is straightforward; JavaScript requires a real browser; infinite scroll requires the program to reason about when to stop.

There is also a useful cross-check: Rungs 2 and 3 scrape the same 100 quotes through two different interfaces—pagination and infinite scroll—and produce matching results row for row. That gives us a completeness check we can actually diff.

There are tradeoffs. Setup required a ~110 MB Firefox download and some Windows environment fixes. The Claude Code plugin also uses headless Firefox and its own screenshot capabilities rather than Webwright’s standard Chromium setup. For a one-off click, this approach is overkill. The payoff appears when tasks involve repetition, dynamic content, or results you need to verify and reuse.

5. Conclusion

Overall, these three tests show that browser-based code generation is more than a way to automate clicks. The final result is a reusable Playwright program that can be run again without the agent. As the websites became more complex, the generated code also became more capable. It moved from simple page loops to rendering JavaScript and finally to reasoning about when an infinite scroll had finished.

The experiments also show the value of verification. The agent found bugs in its own code, questioned incorrect assumptions in the task description, and confirmed that the extracted data was complete. The matching results from the paginated and infinite-scroll versions of the quotes site provide extra confidence that nothing was missed. This is difficult to achieve with a one-time browser recording alone.

There are costs. Running Playwright requires a browser download and more setup than simple HTTP scraping or click recording. The generated programs are also longer and require some technical knowledge to understand. However, those costs are outweighed when the task needs to be repeated, maintained, or verified. Overall, the tests suggest that browser-based code generation is a practical approach for building reliable web scrapers that can adapt to different website designs while producing code that is reusable, transparent, and easy to test.

Webwright’s contribution is not a bigger model or a better prompt. It is a simpler idea:

Give the model a terminal, let it program the browser, and keep the result as reusable code.

That changes the web-agent loop. Instead of fragile clicks and constant replanning, the agent can write, run, debug, and reuse a program.

The idea extends beyond browsers. When a model can code and its environment can execute that code, it may be better to write the program that performs the task than predict every action one step at a time.

The best web agents don’t just click. They write the tool—and leave it behind.

6. Sources

Webwright (primary)

  • Microsoft Research — Webwright: A Terminal Is All You Need For Web Agents: https://www.microsoft.com/en-us/research/articles/webwright-a-terminal-is-all-you-need-for-web-agents/
  • GitHub — microsoft/Webwright(README + source, MIT License): https://github.com/microsoft/Webwright
  • Project landing page: https://microsoft.github.io/Webwright/
  • Benchmarks (Online-Mind2Web, Odysseys) as reported by the Webwright team (AutoEval / LLM-judged).

The landscape

  • CodeAct — Executable Code Actions Elicit Better LLM Agents(ICML 2024): https://arxiv.org/abs/2402.01030
  • OpenAI Operator / Computer-Using Agent: https://openai.com/index/introducing-operator/
  • WebVoyager (multimodal DOM+vision agent): https://arxiv.org/abs/2401.13919
  • Set-of-Mark prompting: https://github.com/microsoft/SoM · VisualWebArena: https://arxiv.org/html/2401.13649v2
  • Mind2Web / MindAct: https://arxiv.org/abs/2306.06070 · WebArena: https://webarena.dev/ · AutoWebGLM: https://arxiv.org/html/2404.03648v2
  • Open-source frameworks — browser-use: https://github.com/browser-use/browser-use · Skyvern: https://github.com/Skyvern-AI/skyvern · Stagehand: https://github.com/browserbase/stagehand
  • Playwright: https://playwright.dev/

Diagrams: author-created (matplotlib). Family animations: author-created. Screenshots and generated CLIs