Prerequisites: These tests must exist locally as YAML files.
momentic results upload
command.
- [tests…]: The tests to run. If not specified, selects all tests in the current project. You can specify multiple tests separated by spaces, test paths, or folder paths. You can also pass substrings to match test names.
Options
-c, --config <config>
Path to the Momentic configuration file. If not specified, it defaults to
momentic.config.yaml in the current directory.
--disable-cache
Disable caching entirely. This will cause all steps to run without cached data,
and not save caches after runs. This can result in slower test execution, but
ensures that all steps are run fresh without stored data.
-f, --filter <filter>
Only applicable when using Workspaces. Loads the project with
the specified name.
--custom-headers <customHeaders...>
Custom headers to include in the test run. You can specify multiple headers by
separating them with spaces.
--input-csv <inputCsv>
Path to a CSV file containing input data for the tests. Each row in the CSV file
will be used as input for a separate test run.
--env <env>
The base URL set on the test will always take precedence over the environment.
If you want to override the base URL set on the test, use the
--url-override
option.--url-override <urlOverride>
The URL to override the base URL set on the test or the environment. This is
useful if you want to run the test against a different URL than the one
specified in the test or environment.
--save-cache
Always save updated step caches after successful test runs, even on the main Git
branch and protected Git branches. Please see the
cache saving eligibility documentation
for more information.
--regenerate-cache
Run the test without using any existing caches. Running with this option will
still save step caches at the end of the run. This can be useful for updating
caches after changing a configuration option.
Warning: using this option will cause all steps to run without any cached data,
resulting in significantly longer execution times. This option should not be
used on an ongoing basis.
--start <start>
A command to run before starting the test. This can be used to start a local
server or any other command that needs to be run before the test starts.
--upload-results
Upload test results to Momentic cloud after the test run. This is the same as
running momentic results upload <outputDir> after the test run, where
<outputDir> is the directory specified by the --output-dir option or
./test-results if not specified.
--wait-on <waitOn>
The resource to wait for before starting the test. This can be a URL, a file, or
any other resource that can be waited on. This is useful for ensuring that the
resource is available before starting the test.
--wait-on-proxy <waitOnProxy>
HTTP proxy to use when polling the --wait-on resource. Provide the full proxy
URL, including protocol, optional credentials, host, and port.
--wait-on-timeout <waitOnTimeout>
The timeout for the --wait-on option. If the resource is not available within
this time, the test will fail.
--retries <retries>
The number of times to retry a test if it fails.
-p, --parallel <parallel>
The number of tests to run in parallel. This can be useful for speeding up test
execution, especially for large test suites. Each test will start its own
browser instance.
--timeout-minutes <timeoutMinutes>
Maximum number of minutes to run tests before stopping. When the timeout is
reached, any currently running tests will be stopped, results will be written to
disk, and a summary will be printed. The process will exit with code 1. This is
useful for CI pipelines where you want to enforce a maximum execution time.
--labels <labels...>
Run tests with the specified labels. This allows you to filter tests based on
assigned labels.
--update-golden-files
Update the golden files for Visual diff.
--reporter <reporter>
Specify the reporter to use for the output.
Options:
junitallure-jsonplaywright-json
--reporter-dir <reporterDir>
Specify the directory where the reporter output will be saved. If not specified,
it defaults to ./reports.
--output-dir <outputDir>
Directory to store run artifacts such as screenshots, results, and logs. This
directory will be cleared at the start of execution. If not specified, it
defaults to ./test-results.
--shard-index <shardIndex>
The index of the shard to run. This is used in conjunction with --shard-count
to determine which tests to run in this shard. This is useful for running tests
in parallel across multiple machines or processes. The index starts at 1.
--shard-count <shardCount>
The total number of shards to run. This is used in conjunction with
--shard-index to determine which tests to run in this shard. For example, if
you have 3 shards and you set --shard-index to 1, only the tests that belong
to the first shard will be executed.
--include <includePatterns...>
Only include tests that match the provided regex patterns. Multiple patterns can
be provided. The patterns will be matched against the test file paths and the
pattern only needs to match a part of the path for the test to be included.
--exclude <excludePatterns...>
The opposite of --include: a test that matches any of the provided regex
patterns will be excluded from running.
--skip-quarantined
Skip quarantined tests entirely. By default, quarantined tests will run, but
their statuses won’t impact pipeline success or the exit code of the process.
--only-quarantined
Run only quarantined tests. Only quarantined tests will be executed and their
statuses will impact the pipeline success and the exit code of the process.
--ignore-quarantine
Run all tests and apply their statuses to the pipeline success and exit code.
--record-video
Record videos of test runs. Enabling this option will increase the size of
results. Once results are uploaded, the videos can be viewed in the run viewer.
In order to use this flag, ffmpeg must be installed on the machine. This can be
done using the momentic install-browsers ffmpeg command.