--- title: "Testing Must Be Token Efficient" description: "Teams repriced testing from wages to tokens, and most of those tokens return nothing. Here is why verification spend buys less than it should, the four criteria that separate well-spent tokens from waste, and the teams already testing token efficiently." canonical: "https://momentic.ai/blog/testing-must-be-token-efficient" last-updated: "2026-09-09T02:41:27Z" --- # Testing Must Be Token Efficient URL: https://momentic.ai/blog/testing-must-be-token-efficient [blog](/blog) [/ quality](/blog/category/quality) / testing-must-be-token-efficient Quality Teams repriced testing from wages to tokens, and most of those tokens return nothing. Here is why verification spend buys less than it should, the four criteria that separate well-spent tokens from waste, and the teams already testing token efficiently. Wei-Wei Wu CEO, Momentic Tokenmaxxing. It is where the whole world has lived in the past six months. If you want to keep up with the teams shipping the most code with the fewest people, you throw tokens at the problem. It's a viable strategy. But it has two obvious problems. One, token prices are going up. If you want to tokenmax, your options are pay the frontier labs more and more, or move to less-capable models. Two, tokenmaxxing isn’t a solution to all life's problems. It might work (for now) for code generation. A generation token _shows_ you what it bought. It has a clear outcome. The agent writes the code, and you get a PR and a diff. But verification is a different beast. Spend the same token there, and it's not clear what you got. There's no diff to read. The agent ran some checks and says things look fine, and you can't tell from the outside whether that's real coverage or an expensive way of clicking around. The only way to find out is to check the work yourself, and a result you re-verify by hand returned nothing, whatever it cost. Every token has to count, and in testing, most don't. Testing tokens are badly spent. The future is testing priced in outcomes, where every verification token comes back as coverage, a finding, or confidence you can act on. Testing must be token efficient. ## The past: testing priced in wages Testing has always been something you paid people to do. That cost came as staff engineer review hours, an internal QA team on salary, or a vendor running regressions by hand. However you bought it, you were buying human judgment. Someone who knew what to test, could read a failure, and could tell you whether it mattered. That worked because code showed up at human speed too. One person could roughly check what another person wrote; the two costs rose together, and nobody had to think about what a unit of testing cost. Coding agents broke the ratio last year, and tokenmaxxing is the inevitable end state. One engineer plus agents now merges more in a day than a review team can read in a month. It doesn't matter what you pay; there aren't enough human hours to close that gap. Wages don't fail on cost. They fail on volume. - At human speed, an engineer merges a couple of PRs a day. If each review takes 25 minutes, checking a teammate's work costs under an hour a day. - A full manual regression pass was a working day in itself. [Quora's took seven hours a day](https://momentic.ai/comparison/momentic-vs-browserstack), across more than 500 test cases. - At agent speed, that same engineer merges 20 PRs a day. The same 25-minute review is now over eight hours, so that's one full-time verifier per engineer, just to tread water. - And the seven-hour pass still has to run, over more code, more often. So teams did the obvious tokenmaxxing thing and repriced testing in tokens. That solved the volume problem, and it created the efficiency problem. ## The present: testing priced in tokens Tokenmaxxed testing is a variant of what you're probably already doing. You use the same Claude Code or Codex session that wrote the change to [drive a browser to check it](https://momentic.ai/docs/comparisons/coding-agents), or write some Playwright to cover it. For a demo, this is great. Tests are immediate, because authoring is the one line a coding agent is genuinely good at. The coding agent is the right tool for the change it's writing right now. Tokens go to the moon for five reasons: 1. Nothing persists between sessions. The agent re-learns your checkout flow every single run, and none of that exploration turns into coverage. 2. Every run plans a new trajectory, so two runs of the same brief return different coverage. You can't compare this week's pass to last week's. 3. Flaky results get re-run, which pays twice for the same information. 4. Every UI change sends the agent back to repair selectors and regenerate tests. Tokens go into keeping the suite alive instead of finding bugs. 5. The agent that maintains the suite also debugs it, guessing which selector moved before it gets anywhere near the product bug. Then a person reads the output anyway, and wage-priced triage is back. And the bill grows with your output. More merges mean more UI change, more repair, more failures, and more triage. The waste scales with your velocity. What does building it yourself actually cost? For [a 25-developer team at 80% coverage](https://momentic.ai/blog/build-vs-buy-test-automation), the numbers break down like this: - About $887,000 a year all in. - $548,000 of it, 62%, is engineers investigating failures, at a 1.5% daily flake rate Google itself published. - Writing the tests is 21%, and it's the only line a coding agent cuts. - The expensive line is judgment. Is this failure a bug, a changed selector, or a slow runner? Agents don't answer that on their own. That 62% is the wage bill sneaking back into the token bill. Some teams end up paying both bills at once, running an internal agent while keeping the old vendor, or moving their engineering overhead to testing. The reason for that is trust. A verification layer you don't fully trust returns less than nothing, because you still check its checks and you still pay as if it doesn’t exist. The tokens are spent either way. The return only exists if the result holds up on its own. And no, you can't cap your way out of this. Reasoning-effort settings only set a default, and Raindrop found a two-sentence ["think hard" prompt at low effort used more tokens than the high setting without it](https://www.raindrop.ai/blog/think-harder/). The fix is to spend tokens _efficiently_, where they return something, not to spend fewer of them. ## The future: testing priced in outcomes [Token-efficiency](https://resolve.ai/resources/video/token-efficiency) means your token spend returns something durable, a trustworthy result, or evidence a human can act on. Whether it does depends on the system around the model, since [the same token buys different amounts of work depending on the product it runs through](https://a16z.com/you-are-not-a-model-dont-price-per-token/). Everyone runs on the same few frontier models. The difference is what's cached, what's remembered, what's isolated, and what happens to a failure. Waiting for a better model doesn't fix this. The next model writes better tests and still forgets your product when the session ends. In token-efficient testing, four criteria separate well-spent verification tokens from waste: - What the agent learns becomes durable regression coverage, so the next run costs less than the last. - Results are trustworthy the first time, because execution is isolated and repeatable, so a failure is a finding rather than a prompt to re-run. - A failure comes with reproduction steps and enough evidence for an engineer to go straight to the fix. - Your engineers never maintain the tests. Nobody updates selectors after a redesign or fixes a broken spec, because the suite isn't yours to keep working. A badly spent token buys one run and nothing after it. A well-spent token leaves coverage and product knowledge behind, so each run makes the next one cheaper instead of costing the same again. It also changes what you own. Buy a testing tool and the tests are yours, along with all the maintenance. Buy the outcome, and the tests belong to the provider, who is the one party with a direct reason to keep them cheap to run. Momentic is built around exactly this. The efficiency comes from three mechanisms: - Successful steps get cached and replayed with no AI call, so an unchanged flow costs almost nothing to re-verify. The cache also knows when a page change actually affects a test, so a changed flow still gets a full check. - The agent builds a knowledge base as tests are created and run, so it remembers how your product works instead of rediscovering it every session. - A broken test doesn't get re-patched forever. Runtime AI can fix a broken step, but on its own you pay for that fix on every run. With the cache, the fix from a successful run sticks. When an agent thinks it found a bug, a separate reproducer agent starts from a clean session and tries to hit it again. The agent that found the bug never decides it's real; only a reproduced bug reaches the report. [One session ran 116 agents against a single app and returned 27 bugs](https://momentic.ai/docs/mo/why-mo), two days of manual QA in one run. ## The teams already token-efficient - GPTZero cut retries by 90% and moved its pass rate from 87% to 98.5%. Then it made test coverage an official engineering goal. - Coframe ships 20 PRs a day with one engineer. Its agents create tests over MCP and CI blocks and merge on the results. Verification keeps pace with the code. - Roo built a failure-analysis agent on top of its testing that classifies each failure as a bug, a flake, or a test needing an update, and deploys automatically when confidence is high. That's the expensive line from the cost model: the 62% spent investigating failures, moved from wages to tokens. None of them hired a QA team to get there. For a tokenmaxxing team, testing has to be token-efficient to keep up. ## Keep reading. [Quality The Harness Will Not Save You AI "harnesses" are essential, but not enough to prevent bugs. Teams need to build an independent, behavioral verification layer that tests your software the way a real user would. Wei-Wei Wu 4 min read](/blog/the-harness-will-not-save-you)[Quality The Set Theory of Agentic Testing Why agentic testing isn't just another form of AI-assisted authoring, and how applying set theory helps you identify which tests belong in an agentic loop. Wei-Wei Wu 7 min read](/blog/set-theory-of-agentic-testing)[Quality Speed at the Cost of Quality AI coding tools boost velocity but add debt. Learn why behavioral tests and truth-driven development keep quality high at Cursor speed. Wei-Wei Wu 6 min read](/blog/speed-at-the-cost-of-quality) ## Close the feedback loop. Point Momentic at your app. Free to start, no credit card. [Try for free](https://app.momentic.ai/signup) [Contact sales](/sales)