---
title: "How to Fix Flaky Tests: Root Causes, Fixes, and What to Automate"
description: "Fix flaky tests by removing shared state, timing waits, order dependence and environment drift, with fixes for Playwright, Cypress and Selenium suites."
canonical: "https://momentic.ai/blog/how-to-fix-flaky-tests"
last-updated: "2026-09-22T19:14:34Z"
---

# How to Fix Flaky Tests: Root Causes, Fixes, and What to Automate

URL: https://momentic.ai/blog/how-to-fix-flaky-tests

[blog](/blog) [/ resources](/blog/category/resources)  / how-to-fix-flaky-tests

Resources

A flaky test passes and fails on the same code. Here are the four root causes, the fix for each one, and the parts a testing tool can repair for you.

Wei-Wei Wu

CEO, Momentic

A flaky test passes and fails on the same commit. Nothing in the product changed between the two runs. The test changed its mind.

Teams pay for this twice. First in reruns, then in trust. Once engineers learn that a red run means nothing, a real regression gets the same shrug as a stale selector.

This guide sorts flakiness by root cause, because each cause has a different fix. Retries help with one of them and hide the other three.

## What causes flaky tests

Four causes explain almost every flaky failure in an end-to-end suite.

### 1. Timing and race conditions

The test acts before the application is ready. A button exists in the DOM but its click handler is not attached. A row renders before the fetch resolves. A fixed sleep passes on a fast machine and fails on a loaded CI runner.

Fix: wait for the state you need, not for a number of milliseconds. Wait for the network request, the disabled attribute, or the text you expect. Delete every fixed sleep you own.

### 2. Locators that describe the markup

A selector such as div.card > button:nth-child(2) records where an element sat on the day the test was written. A class rename or a new wrapper breaks it, and the failure says only that the element was not found.

Fix: address elements the way a user finds them, by role, label, or visible text. Where the product gives no stable handle, add a test id in the application code and treat it as an interface.

### 3. Shared state between tests

Two tests use the same account. One test leaves a modal open, a filter applied, or a record behind. The failure follows test order, so it appears when the suite runs in parallel and disappears when you run the file alone.

Fix: give each test its own data and its own session. Create the record the test needs at the start of the test, and never depend on a record another test made.

### 4. The environment under the test

A third-party script times out. A staging deploy lands mid-run. A CI container gets less CPU than your laptop, so animations and timeouts behave differently.

Fix: stub third-party calls that are not part of the assertion, pin the environment the suite runs against, and read run history before you blame the test.

## How to tell a flaky test from a real bug

Do this before you change the test, because the cheapest wrong move is to make a genuine failure quiet.

- Rerun the same commit. A pass on rerun means the test is unstable, not the product.
- Read the failure evidence, not the stack trace. A screenshot at the failing step usually names the cause.
- Check the failure rate across the last thirty runs. One failure in fifty is a different problem from one in three.
- Run the file alone. A pass alone and a failure in the suite points to shared state.

## How to find flaky tests in CI

Find flaky tests by rerunning the same commit and recording every test that changes result. A failure rate per test over the last thirty runs tells you what to fix first. Do it in four steps:

1. **Rerun on the same commit.**Run the suite two or three times on one commit in CI. A test that passes and fails on the same code is flaky, and a test that fails every time is a real failure.
2. **Store the result per test, not per run.**Most CI systems keep the JUnit XML of each run. Collect the status of each test name across runs and compute a failure rate.
3. **Rank by failure rate and by blast radius.**A test that fails one run in five and sits on the merge gate costs more than a test that fails one run in fifty in a nightly job.
4. **Quarantine, then fix.**Move the worst tests out of the gate so a red run means something again. Keep them running so you still see the failure rate fall as you fix them.

Momentic writes a report for every run. Each failed step holds a screenshot, the page state and the reason the check did not pass, so the rerun tells you which cause you are looking at before you open the test.

## How to fix flaky mobile tests

Fix flaky mobile tests the same way as web tests, with two more causes: the emulator or simulator, and the app install. A cold emulator boot, a slow app install, a permission dialog or a keyboard that covers the target makes a mobile step fail on one run and pass on the next.

- Wait for the screen, not for a sleep. Assert that the screen you need is visible before the next tap, and let the runner retry the lookup until the element appears.
- Reset the app state between tests. Reinstall or clear the app data so a test does not depend on the login or the cart the previous test left behind.
- Pin the device. Run against one emulator image and one OS version in CI, and treat a device matrix as a separate job.
- Describe the element, not its id. An accessibility id that a release renames breaks every test that used it. A step that names the element the way a user sees it survives the rename.

Momentic mobile tests run against an iOS simulator or an Android emulator from the momentic-mobile CLI, and steps target an element by description. The same run flags work as on the web CLI, so one CI job runs both.

## How to fix a test automatically when the app changes

A test heals automatically when the runner resolves each step from its intent at run time and proposes a repair when the target moved. This covers the locator cause, which is the largest source of maintenance work, and it does not cover shared state or a broken environment.

In Momentic a step says what the user does, such as click the "Checkout" button, and the runner finds that button on the current page. When a change moves the target beyond what the description resolves, the repair lands as a diff in the YAML test file, and you review it in the pull request like any other change to the suite. A repair you cannot read is a change to the suite that nobody approved.

| Cause | Share of flaky failures in a typical end-to-end suite | Who fixes it |
| --- | --- | --- |
| Timing and race conditions | Largest | The runner, by waiting for state |
| Locators tied to markup | Second | The runner, by resolving intent and proposing a repair |
| Shared state between tests | Smaller, but hardest to see | You, by isolating data and sessions |
| Environment under test | Varies by team | You, by stubbing and pinning |

## What a testing tool can repair for you

Two of the four causes are structural, and no tool fixes them for you: shared state is a test design problem, and a broken environment is an infrastructure problem. Timing and locators are different. Both are mechanical, and both can be handled at run time.

Momentic tests are plain YAML in your repository. Steps describe intent, so a step says `click the "Checkout" button`, not a CSS path. The runner resolves that intent against the live page on every run, which removes the whole class of failures that a markup change causes. When a target still moves, [self-healing](https://momentic.ai/blog/self-healing-test-automation-guide) repairs the step and the repair arrives as a diff you review in a pull request.

Timing is handled the same way. The runner waits for the page to settle before it acts, so the suite does not need fixed sleeps.

`steps:  - action: navigate  url: "{{env.BASE_URL}}/cart"  - action: ai_action  description: click the "Checkout" button  - action: ai_check  description: the order confirmation shows an order number`

An unresolved flaky test does not have to block the merge queue while you work on it. Quarantine keeps it running and keeps its result out of the gate, so you still collect evidence.

## Self-healing approaches, side by side

Every vendor in this space says self-healing. The differences that matter are where the test lives and who approves a repair.

| Tool | Where the test lives | How it handles a moved element |
| --- | --- | --- |
| Momentic | YAML in your repository | Intent-based steps resolved at run time, plus healing that lands as a reviewable diff |
| Autonoma | Vendor platform | Managed AI agents maintain the tests for you |
| QA Wolf | Vendor platform, with Playwright code you can export | A managed service fixes broken tests as part of the contract |
| Functionize | Vendor platform | ML models pick a replacement element from historical run data |
| Maestro | YAML in your repository, mobile only | Built-in waits and retries; there is no AI repair of a step |
| Playwright | TypeScript in your repository | Auto-waiting and web-first assertions; you write the new locator yourself |

Read the middle column first. A repair you cannot read in a pull request is a change to your test suite that no engineer approved.

## A plan for a suite that is already flaky

- Measure first. Rank every test by failure rate over the last thirty runs.
- Quarantine the worst offenders so the gate becomes trustworthy again today.
- Fix by cause, top of the list down. Timing and locators first, because they are the cheapest.
- Delete tests that assert nothing anyone reads. A flaky test with no owner is a test to remove.
- Keep the gate honest. A green run must mean the product works, or the suite goes back to being decoration.

Momentic runs the same tests locally and in CI with `npx momentic run`, and the exit code gates the merge. The [quickstart](https://momentic.ai/docs/quickstart/web) takes about ten minutes.

Related: [the Roo customer story](https://momentic.ai/customers/roo) — 200+ engineering hours back per quarter after moving off a flaky Cypress suite.

## Flaky tests, answered.

What causes flaky tests?    Four causes cover almost all of them. Timing and race conditions, locators tied to markup, state shared between tests, and an unstable environment. Each cause has its own fix, so sort the failure before you change the test.    How do I fix flaky tests in CI?    Start with the failure rate over the last thirty runs, not with the newest red build. Quarantine the worst tests so the gate is trustworthy, then fix by cause: wait for state instead of time, address elements by role or text, and give each test its own data.    Do retries fix flaky tests?    Retries hide them. A retry turns a timing failure green and turns a real regression green as well. Use retries to keep the pipeline moving, and treat every retried test as a bug with an owner.    Is a flaky test always the test's fault?    No. A race condition in the application shows up first as a flaky test. Read the evidence at the failing step before you edit the test, because a genuine bug that you make quiet reaches your users.    What is flaky test detection?    It is the practice of running the same commit more than once and recording which tests change result. The output is a failure rate per test, which tells you what to fix first and what to quarantine.    Can self-healing tests remove flakiness?    They remove the mechanical part. Healing repairs a step whose target moved, and intent-based steps stop a markup change from breaking a run. Shared state and a broken environment stay your work.

Still have additional questions?

## Keep reading.

[Resources   Best Visual Regression Testing Tools: 10 Compared for 2026     Ten visual regression testing tools compared for 2026: Applitools, Percy, Chromatic, BackstopJS, Argos, Storybook test runner, Playwright toHaveScreenshot, Lost Pixel, Meticulous and Momentic, with pricing model, CI integration and diffing method for each.     Wei-Wei Wu     12 min read](/blog/best-visual-regression-testing-tools)[Resources   QA Release Checklist: 12 Steps Before Every Release     A 12-step QA release checklist with an owner and a piece of evidence for each step, and a clear answer to who owns the checklist: engineering or QA.     Wei-Wei Wu     13 min read](/blog/qa-release-checklist)[Resources   Best Puppeteer Alternatives for Browser Automation     Compare the best Puppeteer alternatives for browser automation, E2E testing, and web scraping. Explore Playwright, Selenium, Cypress, Momentic, and more.     Wei-Wei Wu     8 min read](/blog/puppeteer-alternatives)

## Close the feedback loop.

Point Momentic at your app. Free to start, no credit card.

[Try for free](https://app.momentic.ai/signup) [Contact sales](/sales)