Performance

Latency benchmarks

Runtime depends on how a test is written and the state of the system under test. A Momentic test that only aims for feature parity with Playwright or Cypress runs at about the same speed. Thanks to step caching, over 99% of steps execute in under 500ms.

Step latency

Preset action	Average runtime
Click	250ms
Type	340ms
Choosing from a `<select>` element	275ms
Pressing a key	<5ms
Scroll	<5ms
Page check attempt	220ms
Element check attempt	210ms
Visual diff	620ms

AI-enhanced steps are slower on their first run but most are cached for later runs.

AI-enhanced action	First-run runtime range
Locating an element	4-8 seconds
Evaluating an assertion once	5-8 seconds
Extracting data from the page	5-8 seconds
Generating a single command in an AI action	6-12 seconds
Classifying a test failure	20-30 seconds
Auto-healing a section	30+ seconds

Momentic vs. Playwright

We benchmarked Momentic against Playwright using the same login flow in this public test environment. Cached Momentic steps were 52ms slower on average than comparable Playwright functions. Non-cached steps that require AI were 6354ms slower on average. Over 99% of steps executed on Momentic are cached. This benchmark does not cover every Momentic step type. Several AI-native steps, including AI check and Visual diff, do not have direct Playwright or Cypress equivalents.

Method

We built a Momentic test and an equivalent Playwright script that perform the same login flow. We measured three execution modes:

Steps only measures only the time spent executing steps.
End-to-end includes Momentic’s fixed bootstrap and result-upload time. The Playwright measurement includes CLI initialization but no result upload.
First-run disables caching and includes four fresh AI completions.

Measurements ran on an M3 Max MacBook Pro with 36GB RAM running macOS Sonoma.

Results

All values are P50 milliseconds.

	Playwright	Momentic
Steps only	961ms	1173ms
End-to-end	1870ms	3998ms
First-run steps only	N/A	26379ms

The source used for this benchmark is provided below:

fileType: momentic/test/v2
id: log-in-practice-test-automation
url: https://practicetestautomation.com/practice-test-login/
retries: 1
steps:
  - type:
      text: student
      into: the username input
  - type:
      text: Password123
      into: the password input
  - click: the submit button
  - checkPageContains: Logged In

import { chromium } from "playwright";

async function main() {
  console.log("starting steps", Date.now());
  const browser = await chromium.launch({
    headless: true,
  });
  const context = await browser.newContext();
  const page = await context.newPage();
  await page.goto("https://practicetestautomation.com/practice-test-login/");
  await page.locator("#username").fill("student");
  await page.locator("#password").fill("Password123");
  await page.locator("#submit").click();
  await page.getByText("Logged In Successfully").waitFor({ state: "visible" });
  console.log("done", Date.now());
  process.exit(0);
}

void main();

Parallelism and sharding

By default, the CLI runs tests one at a time on a single machine. Use parallelism, sharding, or both to reduce the wall-clock time of a large suite.

Parallelism on one machine: --parallel <n> runs n tests at once, each in its own browser instance. Raise it until the runner’s CPU or memory is saturated.
npx momentic run --parallel 4
Sharding across machines: --shard-count splits the suite across runners, and --shard-index selects the current runner’s shard. Merge the outputs with momentic results merge before uploading.
# runner 1 of 4 npx momentic run --shard-index 1 --shard-count 4 --output-dir test-results/shard-1

You can shard across CI runners and use --parallel within each runner.

Four shards with --parallel 4 can run up to 16 tests at once. See the GitHub Actions guide for a CI matrix and merge workflow, and the run reference for every flag.

Make sure step caching is saving in CI. Cached steps run in well under a second, so a warm suite is much faster than a cold one. See Saving caches for setup.

Mobile test suites

Mobile runs use the same --parallel, --shard-count, and --shard-index flags through momentic-mobile, with a few platform constraints:

--parallel AUTO saturates the current shard. On remote emulators, each test gets its own session and organization quotas are enforced server-side.
Local Android runs need a distinct AVD per parallel test. Multiple tests cannot share the same --local-avd-id.
Local iOS runs should use --parallel 1 because concurrent Appium instances share a driver manifest.

npx momentic-mobile run --shard-index 1 --shard-count 4 --parallel AUTO

Merge shards with momentic-mobile results merge before uploading. See the momentic-mobile run reference for every flag.

Get started

Author

Run

Learn

Platforms

Integrations

Advanced

Migrations

Account

Latency benchmarks

Step latency

Momentic vs. Playwright

Method

Results

Parallelism and sharding

Mobile test suites

​Latency benchmarks

​Step latency

​Momentic vs. Playwright

​Method

​Results

​Parallelism and sharding

​Mobile test suites

Latency benchmarks

Step latency

Momentic vs. Playwright

Method

Results

Parallelism and sharding

Mobile test suites