Skip to content
Back to Blog

tester-army e2e turns agent clicks into test lines

I ran tester-army's e2e v0.22.0 on a small demo-booking page. Plain tests needed no model, and its MCP server returned a test line for every action.

Kartikay Dhar

Head of Product at Sprouts.ai

tester-army e2e turns agent clicks into test lines

My deliberately wrong test failed in 5.25 seconds with the exact text it saw: expected "Routed to Enterprise AE", observed "Routed to SDR queue". That came from tester-army/e2e v0.22.0, an open-source end-to-end testing framework, run against a small "Book a demo" page. The three honest tests passed in about 0.2 seconds each, with no model configured at all, and its MCP server handed back a ready test line for every action.

That is what a verification layer for coding agents looks like in practice: a runner that tells you precisely what broke, plus a live session an agent can drive and turn into a test. Most of it works before you connect any model.

What I tested

  • e2e@0.22.0 with the web engine @e2e-dev/web@0.15.4, on Node.js 24.22.2. The package needs ^22.22.3 || >=24.8.0.
  • A one-file static page: work email, company size and a "Request demo" button that writes a routing result into a status element.
  • Four plain tests (no agent steps): page opens, personal email rejected, 500+ routed to Enterprise AE, and one written to fail.
  • The e2e mcp server, driven by a small MCP client that filled the same form.
  • One agent test with no model set, to see what happens.

Installed, node_modules came to 58 MB.

Plain tests run with no model

My first run failed 4 of 4, and that was my fault. I had written the tests Playwright-style against a browser fixture, and the runner said browser.getByLabel is not a function. In e2e, locators live on screen, and a click is .tap(), since the same API covers mobile. After that change the suite looked like this:

test('routes 500+ to enterprise AE',
async ({ app, screen }) => {
await app.open('/');
await screen.getByLabel('Work email')
.fill('vp@acme.com');
await screen.getByLabel('Company size')
.selectOption('500+');
await screen
.getByRole('button', 'Request demo')
.tap();
await expect(
screen.getByRole('status'))
.toHaveText(
'Routed to Enterprise AE');
});

I ran the fixed suite four times. The numbers below are the runner's own output.

TestFinal runAll four runs
page opens202 ms202–212 ms
rejects personal email210 ms197–240 ms
routes 500+ to enterprise AE215 ms215–257 ms
deliberately wrong (fails)5.25 s5.25–5.38 s
Whole run (Duration)6.42 s6.42–6.63 s
Runner startup95 ms95–97 ms

The failing test sets the run time. Almost all of it is the assertion waiting for the right text before it gives up, which is the behaviour you want from a retrying assertion.

Terminal output of e2e v0.22.0: three passing tests around 200 ms and one failing test at 5.25 s with expected and observed text

How does e2e report a failing test?

The failure output gives the locator, the expected text, the observed text and a path to a trace.md for that test. The trace is the part I would hand to a coding agent. It lists every step with its source line and timing, what changed on screen after each one, and why the last step failed: "expect gave up after 48 reads in 5.0s".

4. ✓ locator.tap
getByRole("button",
name: "Request demo") (31ms)
- added status
"Routed to SDR queue"
5. ✗ expect.toHaveText
getByRole("status")
(5.0s) ASSERTION_FAILED

It ends with the screen at the moment of failure, both as a screenshot and as a text tree of nodes with roles, names and values. A model can read the tree without looking at pixels.

Screenshot e2e captured at failure: the Book a demo form with 1-50 selected and the status Routed to SDR queue

The trace also records the browser: Chromium 156.0.8078.4, headless. Error codes are typed too. ASSERTION_FAILED is a test result, while a missing model is a configuration error, so a script or agent can tell "the app is wrong" from "the setup is wrong".

What the MCP server gives a coding agent

The e2e mcp server exposed four tools: open_session, tools, call and close_session. Once a session was open, tools listed 28 actions such as observe, tap, type, select and press. The docs say the server needs no model, and mine ran without one.

My client opened a session, observed the page, typed an email, picked "500+", tapped the button and observed again. Each action came back with what changed on screen and a test line that matches the test I wrote by hand:

e2e MCP session output: each type, select and tap call returns a ready test line, and the tap adds the status Routed to Enterprise AE

Timings from the same session, measured from when my client started: the server answered initialize at 461 ms, the session was open at 1,198 ms, and the five actions ran between 1,201 and 1,827 ms, about 0.63 seconds in total. close_session reported "5 tool calls ran".

This closes a common gap. Without it, an agent calls a UI change done because it compiles. Here it can open the page, act and leave behind a test that checks the result, which is the idea behind harness engineering: give the agent a way to check its own work.

Where the model comes in

Agent steps such as agent.act() need a model. My agent test with none configured stopped in 82 ms with MODEL_UNAVAILABLE, a configuration error that names the missing setting and shows an example. The whole run took 731 ms. It failed fast and said why, which is better than a hang.

The README says an agent step that a later assertion verifies records its actions, and the next run replays them with no model calls until the app changes. You can bring a subscription, an API key, a local model, or your own coding agent over ACP. Replay is the claim I would most want to watch on a real app, because that decides what a suite costs to run every day. For scoring agents themselves rather than apps, my Microsoft Scope write-up covers a different layer.

What this means for product and GTM teams

The forms that matter most to a business, signup, pricing and demo requests, are the ones that break quietly. A routing rule changes, the page still loads, and the break can sit there unnoticed. A plain e2e test like mine catches that in about a fifth of a second, and its failure reads like a bug report: what was expected, what showed up, and the screen at the time.

For teams shipping with coding agents, the MCP server matters more. The agent that changed the form can check its own work and leave a test behind, so the review is a diff and a trace, not a promise. Start with the tests that need no model. They already catch the breaks that cost the most.

Frequently asked questions

Does e2e need an AI model to run tests?

No. Tests built from locators and assertions ran with no model. Only agent steps such as agent.act() need one, and without it the run stops with MODEL_UNAVAILABLE.

Is e2e a drop-in replacement for Playwright?

Not line for line. It uses Playwright for the browser engine, but tests use a screen fixture and .tap(). My Playwright-style browser locators failed until I switched. The docs include a Playwright migration guide.

Which Node.js version does e2e need?

Node.js 24.8 or newer, or 22.22.3 or newer on Node 22. I used 24.22.2.

What does the e2e MCP server do?

It gives a coding agent a live session on your app through four tools. The agent observes the screen, runs actions like tap and type, and gets back a test line for each action. In my run, five actions took about 0.63 seconds.

Sources

Kartikay Dhar

Head of Product at Sprouts.ai, where he leads One Platform, Sprouts Data Universe and the Sprouts MCP. On Instagram he shares one free GitHub repo a day.

  1. Tool Breakdowns

    ui-diff tested: visual QA that caught a contrast bug

    I ran ui-diff on a two-page demo. It caught a 1.91:1 contrast drop, and Cursor's agent correctly explained a misleading focus-order note.

  2. Tool Breakdowns

    DeepSeek Super-agent is a router for coding work

    DeepSeek Super-agent is a research prototype for a proactive GitHub coding agent on DeepSeek Harness. Here is what the open docs say it is, and is not.

  3. Tool Breakdowns

    agent-dream tested: AI memory that cites its sources

    I ran agent-dream, a free MCP server that turns past coding chats into memory pages. Each fact cites its messages, and a fake citation was rejected.