Keep What You Kill

We recently wrapped up a game-changing POC with a public company. $1.3M/year saved just on engineering hours. 5x faster test creation. 78% improved reliability compared to their old setup.

But here's the thing—those savings aren't the real win. The real win is what happens next.

The AI Dividend Trap

There is an excellent line from Box’s AI-first memo:

“in general, we'll bias towards a "keep what you kill" mentality with AI, which means that when an organization frees up resources or saves money with AI, we'll aim to reinvest those savings back into more strategic things. Again, we want to use AI to get more great work done, not just the same amount as before.”

Beyond being a huge Vin Diesel fan, Box CEO Levie is grokking the genuine opportunity of going all in on AI-first: more great work done. Every company using AI right now faces a choice. You will save money by using AI. The choice is now in what to do with those savings:

1. Pocket the efficiency gains and call it a day: Cut headcount, reduce budgets, optimize for margin.
2. Reinvest every dollar and every hour into growth: Use AI savings to tackle bigger problems, serve more customers, and build the future faster.

Choosing option one isn't necessarily a mistake. There is an obvious virtue in leanness in startup terms. You can extend your runway, weather downturns better, and maintain the scrappy efficiency that got you here. Running lean forces discipline, creativity, and careful prioritization. It's the responsible choice, the prudent choice. It's what most boards would recommend in uncertain times.

But AI isn't just another efficiency tool. It's not like switching from Slack to Teams to save a few bucks per seat. When you treat AI-driven savings like traditional cost-cutting, you play by the old rules in a fundamentally new game.

The companies that win in the AI era won't be the ones who got 30% more efficient. They'll be the ones who used that 30% to do things that were previously impossible.

The Compound Effect of Reinvestment

Say you've got a team of 10 engineers. AI tooling makes them 3x more productive. You can think, "Great, now I need 3.3 engineers instead of 10."

(Good luck finding that 0.3 engineer. I hear they're great at writing documentation but terrible at stand-ups.)

But that’s the wrong math. The correct math is that you now have the equivalent of 30 engineers. What could you build with 30 engineers that you couldn't with 10?

If we model out the POC from above, post-AI implementation, we might get:

• Test creation: 5 hours → 1 hour (5x faster)
• Test maintenance: 40% of sprint → 5%
• Test reliability: 22% flake rate → 4% (78% improvement)

Suddenly, your team is available to do everything else. Product features. Performance optimization. Security hardening. All the work that moves the business forward. Think also about what that $1.3M really represents.

• More engineers. This is 3-5 more engineers you can hire, and they’ll be working on core product.
• Better engineers. You can reinvest to pull in higher-quality, senior developers previously out of budget.
• Better tooling. You get budget for tools across the stack.
• More resources. Infrastructure upgrades, training programs, experimental projects that were previously unfunded.
• More time. Current engineers aren’t locked into testing, so they also have time to work on product.

With AI (testing, coding, or strategy), the first wave is efficiency. You automate the busy work, speed up the repetitive tasks, and eliminate the drudgery. That's table stakes.

The second wave is force multiplication. You apply AI to the work your engineers are doing. Code generation for boilerplate. Automated refactoring. AI-powered debugging. PR reviews that catch real issues. Your freed-up engineers aren't just working on product—they're working 2-3x faster on product.

The third wave is where it gets interesting. You take all that freed-up brainpower and point it at harder problems. More strategic initiatives. Deeper customer insights. Bolder experiments.

The Strategic Advantage of AI-First

When you go AI-first on testing, you're not just automating tests. You're fundamentally changing how your engineering org operates.

Traditional testing is a tax on velocity. Every new feature needs tests written. Every refactor needs tests updated. Every flaky test needs debugging. Your best engineers spend 20-30% of their time on test maintenance. Your QA team is perpetually behind. Your release cycles are gated by test runs that take hours.

AI-first testing flips this. Tests write themselves from PRDs. They update automatically when code changes. They self-heal when the DOM shifts. Your engineers get that 20-30% back immediately. But more importantly, they stop thinking of tests as a burden. Testing becomes invisible infrastructure, like your CI/CD pipeline.

This invisible infrastructure creates compound strategic advantages:

• Faster feedback loops. When tests run in minutes instead of hours, engineers test more frequently. Bugs get caught in development, not staging. Features ship daily instead of weekly. The entire organization starts moving at a different clock speed.
• Higher engineering morale. Nobody became an engineer to write Playwright selectors. When you remove the drudgery, engineers focus on solving real problems. They experiment more. They refactor more confidently. They take on ambitious projects they would have avoided before.
• Strategic flexibility. You can pivot faster when you can spin up comprehensive test coverage in hours instead of weeks. That experimental feature? Test it properly without dedicating a QA engineer for a month. That acquisition? Integrate their codebase without a 6-month test-writing project.
• Quality becomes a differentiator. Most companies choose between velocity and quality. AI-first testing breaks this tradeoff. You ship faster and more reliably. Your competitors are still deciding between moving fast and not breaking things. You're doing both.

The same pattern applies everywhere you go AI-first. Documentation that updates itself. Code reviews that catch actual bugs. Monitoring that predicts issues before they happen. Each implementation frees up time and mental energy for strategic work.

This is why "keep what you kill" matters. Every hour saved is an hour to invest in the next strategic advantage. Every dollar saved is a dollar to spend on pulling further ahead. The companies that understand this aren't just using AI—they're using AI to buy themselves the ability to use AI better.

The Red Queen Effect

In Lewis Carroll's Through the Looking-Glass, the Red Queen tells Alice: "It takes all the running you can do, to keep in the same place."

That's the AI landscape right now. Your competitors aren't pocketing their AI savings either. While you're debating whether to cut headcount or reinvest, they're already deploying their savings into the next wave of AI tools.

Consider the math:

• Company A pockets the savings and becomes 30% more efficient
• Company B reinvests everything, compounds its gains quarterly
• After one year, Company B is shipping 3-4x faster than Company A
• After two years, the gap is unbridgeable

This is the formula: AI creates time, time creates opportunity, opportunity creates advantage. Keep what you kill, and kill what stands between you and the future.