Run tests locally
Run tests in any private environment, including local and CI machines.
This command requires an API key to authenticate against Momentic’s servers.
Overview
Run one or more Momentic tests using the current machine as the browser host. Any AI-enhanced steps are transmitted to Momentic’s servers for analysis. Since the test browser runs on your machine, your Momentic tests can access private environments, including sites only available on localhost, deployed within a private VPC, or gated by VPNs.
Arguments
| Argument | Description | Required |
|---|---|---|
testFilesOrSubstrings | One or more paths to files on disk. You may also pass directory names, which will automatically include all test files under that directory. If any argument provided is not a valid path on disk, the arguments will be interpreted as substrings to be matched against the paths of each test in the active project. | Yes |
Options
| Option | Description | Default |
|---|---|---|
--shard-count <shardCount>. | Split the tests into separate shards for parallelizing when running with the --local flag. Each shard will run 1/shardCount of the total tests. | |
--shard-index <shardIndex>. | Which shard to run when using the --shard-count flag. | |
--pixel-ratio <pixelRatio>. | Momentic features that require calculating relative visual positions are affected by your monitor’s pixel density. If unsure, please visit this site to find your machine’s pixel ratio. | 2 when a MacOS Retina display is detected, otherwise 1 |
--input-csv <inputCsv> | Load test inputs from the specified file when running with the --local flag. | |
--include <includePatterns...> | Only include tests that match the provided regex patterns. Multiple patterns can be provided. The patterns will be matched against the test file paths and the pattern only needs to match a part of the path for the test to be included. | |
--exclude <excludePatterns...> | The opposite of —include: a test that matches any of the provided exclusion patterns will be excluded from running. | |
--no-report | Skip reporting test results to Momentic Cloud when running with the --local flag. | |
--start <start> | Arbitrary setup command that will run before Momentic steps begin. | |
--wait-on <waitOn> | URL to wait to become accessible before Momentic tests begin. | |
--wait-on-timeout <waitOnTimeout> | Max time in seconds to wait for the --wait-on URL to become accessible. | 60 |
--retries <retries> | Number of retries to attempt when running tests locally. Defaults to each test’s own retry configuration. | |
-p, --parallel <parallel> | When running with the --local flag, the number of tests to run in parallel. Defaults to 1. | 1 |
--env <env> | Name of the environment to use when running tests. | |
--url-override <urlOverride> | Fully qualified URL (e.g., https://www.google.com) to start all tests from. Overrides any default starting URL set from the test or environment. | |
--custom-headers HEADER-1=value HEADER-2=value | Add or override HTTP headers to be sent on every network request that occurs during the test. If this option is provided, you must use -- to separate the options from the test names provided (e.g. npx momentic@latest run --custom-headers TEST=true -- my-test). | |
--reporter <junit|allure> | Output standardized reports to disk based on the test results. Junit XMLs can be used with Junit visualizers while Allure JSON files can be bundled into static sites. | |
--reporterDir | Specify the folder to which standardized reports should be written. | reports |
--labels <labels...> | Only run tests that have at least one of the specified labels. Labels can be provided by using this flag multiple times, or as a list separated by spaces. | |
--filter <projectFilter> | Select the active project that should be used for test collection and running. Only applicable when using workspaces. |
Examples
Single project
The following examples apply to teams using a single project and are based off this fictional file structure. Assume all commands are ran from the root of the file system.
Run all tests by folder
Run the entire momentic folder, where all the tests live:
Run all tests by glob
Use built-in Bash globbing to match every test file under the momentic directory, recursively:
Run tests with exclusions
Run all tests except those that whose paths match the regex experimental (i.e. the entire momentic/experimental folder).
Run tests matching a pattern
Run only the tests that have flaky in their path.
Run only the tests whose file names start with v2- and end with referral. Note the single quotes are required to avoid triggering Bash globbing.
Run tests in the experimental folder that do not contain v2 in their path.
Run tests with substring matches
Run all tests containing the substring signup somewhere in their path. The test path is the relative path from the test to the project root (where the momentic.config.yaml resides).
Run tests containing either the substring signup or referral in their path.
Run multiple sets of tests
Run two specific test files:
Run a folder of tests and a specific test file:
Run tests by label
Run tests that have the staging label:
Workspace
Momentic uses the concept of workspaces to allow users to organize multiple projects in the same repository.
To run tests from the workspace level, you must specify which project’s tests are in scope. Only one project can be “active” at any given time.
The following examples are based on the multi-project file structure below. Assume the working directory for all commands is the root of the file system (where the momentic.workspace.yaml resides).
Run all tests in a project
Manually navigate to the project before running tests normally. The momentic.config.yaml is automatically detected.
Or, you can specify the path to the project’s config file directly:
Or, you can specify the project by name. This assumes that the name field in the marketing project’s momentic.config.yaml is set to marketing.
In all three examples above, the argument at the end corresponds to the directory in which Momentic should look for matching tests. Any tests found must also be part of the project specified, according to the project’s configured inclusion and exclusion criteria. As such, it would be equivalent to pass . to indicate anything under the current directory instead.
Run project tests with a substring
Run the billing test in the dashboard project using a substring match:
Run project tests with the full path
Run the billing test in the dashboard project by specifying a valid path:
Run project tests with a pattern
Run the billing test by using a regex. The . at the end is treated the same as any other directory argument and indicates to search for files anywhere under the current directory. Passing apps/dashboard as the last argument would have the same effect.
Was this page helpful?