Skip to content
Back to Blog

agent-browser: how AI agents can check your web app

agent-browser lets AI agents use a real browser. I ran v0.41.2 on Wikipedia and example.com and checked every click myself.

Kartikay Dhar

Head of Product at Sprouts.ai

agent-browser from Vercel Labs is a free, open-source tool that lets an AI agent use a web browser. It reads the page as text and gives buttons and links short labels. Your agent can use those labels to click, fill forms and check the result.

The text view is called a snapshot. Each short label is a ref.

If your team is building with AI agents, this gives them a way to check a working app. A coding agent can open the app it built and try a user task. You can also use a separate test agent to try the agent already inside your product through its web interface. Those are different tests: one checks the app, while the other checks what your product's agent does.

As agents take on work in web products, a message saying "done" is not enough evidence that the task worked. You need to see the expected result. agent-browser supplies browser controls and page reads. Your team still has to choose the task, define a pass and check the outcome. Your team owns the test plan; agent-browser carries out the browser steps.

I ran v0.41.2 against the Wikipedia article on web browsers and against example.com, three fresh sessions each. The Wikipedia snapshot was 548 lines in every run, and clicking the History link by its ref took 0.021 to 0.025 seconds.

What I tested

  • Version: agent-browser 0.41.2, the current npm release, with Chrome for Testing 157.

  • Task: open en.wikipedia.org/wiki/Web_browser, take an interactive snapshot, find the History link in the table of contents, click it by ref, then check the URL and the scroll position myself.

  • Second task: click "Learn more" on example.com by ref, then check URL, title and open tabs.

  • Semantic find: a short set of find queries by role, text and placeholder.

  • Runs: three fresh sessions for each page, with separate timings for page opening and clicks.

How can a product manager start?

You will need a computer that can run terminal commands, which are typed instructions, and an AI agent that is allowed to run them. The tool is free; your AI plan or usage may still cost money. For the npm route below, install Node.js and npm first. Use a test copy of your app with fake data and a test account. Ask an engineer to help with installation, agent access and sign-in if these are new to you. An agent built into your platform will need a way to call this tool; installing a skill alone does not connect it.

The official installation guide gives these commands. The first installs the tool; the second downloads Chrome for Testing. Use them for setup before trying your own task.

npm install -g agent-browser
agent-browser install

On Linux, agent-browser install --with-deps also installs the browser libraries and may need administrator approval. On macOS, Homebrew is another option: brew install agent-browser, then agent-browser install.

For a supported coding assistant, install the skill, a set of instructions for using the tool, with:

npx skills add vercel-labs/agent-browser

Before its first browser task, tell the agent to run agent-browser skills get core and follow the output. These instructions match the installed tool version. Your engineer can also connect it through MCP, a standard way for an AI app to call tools, if your platform uses that.

A suggested first test on your platform

The example below is a starting point for your team, not a task measured in this post. Start with a harmless job, such as creating a draft task with a unique name in a test account. Avoid payments, real emails and customer data.

  1. Write the expected result first: exactly one draft task with that name should exist after the page is reloaded.

  2. To check an app a coding agent built, ask that agent to open the test app and create the draft through its form.

  3. To test an agent already in your product, have a separate test agent open your product's chat, request the draft task, then inspect the task list. This tests the browser-visible result; it does not show every step inside your product's agent.

  4. Have the test agent read the page again after each action and navigation. At the end, check the saved task after a reload. Keep the command log and a screenshot, and have a person review the first result.

Count the test as passed only when the saved result matches your rule. A success message or a click alone does not pass it. If the browser cannot see the saved result, ask an engineer for a separate check. Once that flow is useful, turn it into a repeatable test.

How does the ref loop work?

A command-line tool, or CLI, runs typed commands. agent-browser is written in Rust. It sends each command to a small background process that starts on the first command and controls Chrome. The browser stays open between commands, so it starts only once per session. In my runs, open took 2.03 to 2.13 seconds on Wikipedia, and almost everything after it took milliseconds.

A ref is a short label for an item on the page. This is the whole loop from one of my runs:

agent-browser open \
  https://en.wikipedia.org/wiki/Web_browser
agent-browser snapshot -i
#  - link "History" [ref=e202]
agent-browser click @e202
# ✓ Done
agent-browser get url
# .../Web_browser#History
agent-browser close

The -i flag keeps the snapshot to interactive elements plus the structure around them. On this article it came back as 548 lines and 20,101 bytes, byte-identical across all three runs. That is how much page text the agent received before choosing its next action. A smaller snapshot can reduce how much text the agent needs to read.

What did the Wikipedia runs show?

  • Run 1

    • open: 2.10 s

    • click @e202: 0.025 s

    • Whole run: 2.76 s*

    • Landed on #History: yes

  • Run 2

    • open: 2.13 s

    • click @e202: 0.021 s

    • Whole run: 2.67 s

    • Landed on #History: yes

  • Run 3

    • open: 2.03 s

    • click @e202: 0.024 s

    • Whole run: 2.58 s

    • Landed on #History: yes

*Run 1 also took a screenshot. "Whole run" covers open, two snapshots, click, URL check, a JavaScript check and close.

The History link was @e202 in all three runs. After the click I did not trust the ✓ Done alone. I ran get url, which returned the #History hash, and an eval that read the heading position: 75 px from the top of the viewport, in view, every time.

On example.com the "Learn more" link was @e1. The click returned in 0.016 to 0.019 seconds, and in all three runs the URL, the title "Example Domains" and the single open tab all pointed at iana.org. The click navigated every time.

Why is "Done" not the same as done?

A successful click means the tool found the element and sent the click to the browser. It does not mean the page reached the state the agent wanted. The README is precise about this: a click fails when the target is hidden, disabled, out of view or covered. It says nothing about checking where the click led.

The find tests showed why this matters. find role link click --name History printed ✓ Done and went to /wiki/History_of_the_web_browser, a different link that also contains "History". find role link click --name HTML landed on /wiki/HTML_3. Both reported success while clicking the wrong link for what I meant. Name matching here is loose, so a similar name can lead it to the wrong item.

The two searchbox fills also returned ✓ Done, but I never read the field value back, so I am only claiming the command returned. The text reads worked: the heading and text lookups for "History" both returned "History". One query failed with "No element found", and that was my test design: the page had already moved to iana.org.

After an action, check the result that matters: the destination page, the text in a field or a saved record. The Ralph loop walkthrough also uses browser checks before counting work as complete.

How does it compare with Playwright?

I did not benchmark the two, so this is how they differ on paper. Playwright is Microsoft's browser automation library. You write a script or test in code, and it runs the whole flow. That is the right shape for a repeatable test suite.

agent-browser is shaped for an agent that does not know the page in advance. It reads, picks a ref, acts, reads again, all through short terminal commands. The docs recommend refs to choose which page item to act on. The docs say a ref stays tied to the same page element while it exists. Navigation invalidates old refs. If an element is replaced, the tool may recover it by matching its role and name, but only when the ref was unique in the snapshot. If the match is ambiguous, the action fails. So the safe pattern is to snapshot again after every navigation.

Playwright can serve agents too. Microsoft's Playwright MCP gives models a text view of a page. Its README also points coding agents to Playwright CLI with skills. It says short commands can use fewer tokens, the units of text a model processes, than loading large tool descriptions. agent-browser has an MCP mode too. I would use Playwright for tests that run automatically with code changes, often called CI, and agent-browser for an agent exploring a page through short commands.

Both tools are free under Apache-2.0. Your AI plan or model usage and the computer running it may have separate costs. agent-browser can also use paid cloud browsers such as Browserbase or Browserless.

agent-browser uses Chrome for Testing by default. It can also connect to an existing Chrome or use the Lightpanda engine.

What are the limits?

  • Snapshots can get long: this Wikipedia article returned 548 lines. The agent has to read that text before it acts.

  • Refs go stale after navigation, so the agent has to snapshot again.

  • "Done" only means the command ran. Loose name matching in find can pick a similar link.

  • It drives a real browser. Protect sign-in details and saved sessions as you would in other browser automation. The security guide covers restricting which sites and actions the agent can use.

The snapshot docs offer ways to read less. -s "#main" reads one page section, if it has that identifier. -d 3 limits how many nested levels appear. -c removes empty branches while keeping visible text and status text. -i filters to interactive items, so it can leave out text you need to prove a result. Ask an engineer to choose a scope that still includes your success check. These options were not measured in the runs above.

agent-browser snapshot -s "#main"
agent-browser snapshot -d 3
agent-browser snapshot -c

What this means for product and GTM teams

If your team is giving coding agents a browser to check their own work, the click itself can be quick. In my runs it was a few hundredths of a second. Reading the page and proving the result are separate work. I would budget for how much text the agent reads and require a result check after each action.

Give links and buttons clear, distinct names. In these tests, a request for "History" could match a different link. Names that describe the action help an agent choose the right control. For checks you need to run often, keep explicit pass rules and saved evidence; the tester-army e2e walkthrough shows one test line per action.

Frequently asked questions

Is agent-browser free?

Yes. It is open source under Apache-2.0 and installs from npm or Homebrew. The tool has no licence fee. Your AI agent or model, the computer running it and any paid cloud browser provider may have separate costs.

What is a ref in agent-browser?

A short ID such as @e202 that the snapshot assigns to an element. You pass it to commands like click or fill. It stays valid while that page element exists, so take a new snapshot after navigation.

Does a successful click mean the task worked?

No. It means the click ran. In my tests, semantic find clicks reported success on the wrong link. Check the URL, title or a field value afterwards.

Should I use it instead of Playwright?

For a repeatable test suite, Playwright is the better fit. For an agent exploring a page step by step through terminal commands, agent-browser is built for that loop. I did not benchmark one against the other.

Sources

Kartikay Dhar

Head of Product at Sprouts.ai, where he leads One Platform, Sprouts Data Universe and the Sprouts MCP. On Instagram he shares one free GitHub repo a day.

  1. Tool Breakdowns

    ui-diff tested: visual QA that caught a contrast bug

    I ran ui-diff on a two-page demo. It caught a 1.91:1 contrast drop, and Cursor's agent correctly explained a misleading focus-order note.

  2. Tool Breakdowns

    DeepSeek Super-agent is a router for coding work

    DeepSeek Super-agent is a research prototype for a proactive GitHub coding agent on DeepSeek Harness. Here is what the open docs say it is, and is not.

  3. Tool Breakdowns

    agent-dream tested: AI memory that cites its sources

    I ran agent-dream, a free MCP server that turns past coding chats into memory pages. Each fact cites its messages, and a fake citation was rejected.