Skip to main content

Documentation Index

Fetch the complete documentation index at: https://momentic.ai/docs/llms.txt

Use this file to discover all available pages before exploring further.

​
Execution speed

Runtime depends on how a test is written and the state of the system under test. A Momentic test that only aims for feature parity with Playwright or Cypress runs at about the same speed. Thanks to step caching, over 99% of steps execute in under 500ms:
Preset actionAverage runtime
Click250ms
Type340ms
Choosing from a <select> element275ms
Pressing a key<5ms
Scroll<5ms
Page check attempt220ms
Element check attempt210ms
Visual diff620ms
AI-enhanced steps are slower on first run but most are cached for subsequent runs. Approximate first-time runtimes:
AI-enhanced actionFirst-time runtime range
Locating an element4-8 seconds
Evaluating an assertion once5-8 seconds
Extracting data from the page5-8 seconds
Generating a single command in an AI action6-12 seconds
Classifying a test failure20-30 seconds
Auto-healing a section30+ seconds

​
Benchmarks

​
Overview

We have published a basic benchmark comparing Momentic against Playwright in this publicly accessible test automation environment. The results illustrate that cached Momentic steps are only 52ms slower on average than comparable Playwright functions. Non-cached steps that require AI to execute run on average 6354ms slower. Over 99% of all steps executed on the Momentic platform are cached. Note that this benchmark does not exhaustively test all Momentic step types, many of which do not have analogs in Playwright, Cypress, or any traditional tooling (e.g. AI check, Visual diff).

​
Method

We built a Momentic test that logs into the practice automation site, as well as an equivalent Playwright script that performs the same sequence of actions. We obtained three different sets of measurements:
  • The “Steps only” category only measures the time spent executing steps in both software.
  • The “End-to-end” category includes Momentic’s fixed bootstrap (e.g. API key check) and test result upload times. For Playwright, the end-to-end time includes CLI initialization time but does not involve any upload of data.
  • The “First-run” category ran with caching explicitly disabled and thus includes the runtime of 4 fresh AI completions.
All measurements were completed on a M3 Max Macbook Pro with 36GB RAM running macOS Sonoma.

​
Results

All values are P50 milliseconds measured over 10 independent samples.
PlaywrightMomentic
Steps only961ms1173ms
End-to-end1870ms3998ms
First-run steps onlyN/A26379ms
The source used for this benchmark is provided below: 
fileType: momentic/test/v2
id: log-in-practice-test-automation
url: https://practicetestautomation.com/practice-test-login/
retries: 1
steps:
  - type:
      text: student
      into: the username input
  - type:
      text: Password123
      into: the password input
  - click: the submit button
  - checkPageContains: Logged In