Building an end-to-end suite got cheap in the part everyone measures. A coding agent writes a test in minutes that used to take an afternoon. So the build side of the argument now sounds obvious: the framework is free, the agent is fast, and a vendor is a line item.
The bill does not arrive there. It arrives in the year after, and the largest item in it is not writing tests.
A cost model for an in-house suite
The scenario below is a 25-developer team, 80% coverage of critical flows, one run a day. You can move every input on the calculator.
| Cost | A year | Share |
|---|---|---|
| Investigating failures | $548,000 | 62% |
| Writing the tests | $188,000 | 21% |
| Model inference | $143,000 | 16% |
| Runners and CI | $8,000 | under 1% |
| Total | $887,000 |
That is 625 tests, 162,500 test runs a year, and 2,438 failures to investigate at 1.5 hours each. The 1.5% daily failure rate is the flaky-result rate Google published, not a number chosen to make the case. Move every input yourself .
The line item nobody puts in the plan
Writing tests is 21% of the bill, and a coding agent is already cutting it. Investigating failures is 62%, and an agent does not cut it by itself, because the work is a judgment: is this a real defect, a changed selector, or a slow runner? Somebody senior enough to answer that reads the failure. Do that 2,438 times a year and it is three full-time engineers.
Runners cost under 1%, which is why the "it's just CI minutes" version of the build case is arguing about the wrong line. The plan almost always counts the framework, the runner and the agent, and almost never counts the engineer who investigates Monday's failed run.
Can a coding agent maintain the suite?
For a stretch, yes. An agent repairs a broken selector well, and it writes the replacement test faster than a person.
Where it stops: an agent that repairs the failure it is shown can turn a real defect into a passing test. Deciding which failures deserve a repair, and which deserve a bug, is the expensive half, and it is the half that does not automate on its own. An agent also does not own the emulator fleet, the browser fleet, the retry policy or the quarantine list.
What you are actually buying
- The triage, not the authoring. Failures arrive classified, and a locator that moved is repaired during the run, not by an engineer on Monday.
- Ownership of the tests stays with you. Steps are plain-English YAML in your own repository, reviewed in pull requests. The buy decision does not hand your suite to a vendor's workspace.
- Infrastructure billed on usage: hosted browsers, emulators and simulators, running in parallel, with nothing to operate.
- One suite across web, iOS and Android, rather than three frameworks and three fleets.
When building still makes sense
- Your product is a handful of critical flows and the suite is small enough that a failure a week is a five-minute read.
- You have a hard constraint no vendor meets: an air-gapped network, a regulated environment, or hardware only you own.
- Testing infrastructure is your product, or it is close enough to it that the expertise is an asset rather than a cost.
- You already have a team that runs the fleet well, and the marginal cost of the next suite on it is near zero.
If two of those are true, build, and the calculator will say so.
How to decide in an afternoon
Use the calculator page to test the model with your own failure count. Then compare the result with the pricing page , or talk to sales for a walkthrough.