Failure recovery is in beta and subject to be deprecated in the future.
Recovery behavior
When an eligible test fails, Momentic’s AI will diagnose the test failure and
determine if it is “recoverable”. Recoverable failures are those caused by
explainable, transient issues that can be resolved without modifying the
original intent of the test.
If the test is deemed recoverable, Momentic will generate and execute steps to
restore the test back to a good state. The failed step will then be re-executed.
If successful, the test will continue running normally. If not, the test will be
marked as definitively failed.
All additional steps executed or retried by Momentic will be marked with a
special purple status in the run viewer. If the test is executed using the CLI,
the status line printed by momentic run will also indicate that the test was
automatically recovered.
Configuration
Failure recovery can be enabled or disabled on a per-test basis in test options.
The default for all tests can also be configured in your organization’s
CLI configuration for local executions or on the
AI settings page for Cloud runs.
Failure recovery will use your test’s description to understand the purpose of
the test and generate appropriate recovery steps. In addition, using the test
options modal, you can also describe additional custom scenarios that should be
recovered in a certain way.
Limitations
A single test run is eligible for failure recovery up to 3 times. Only primitive
steps can be recovered: steps that contain other steps such as modules and AI
actions are not eligible.
Only tests running in a CI environment (detected through the CI environment
variable) are eligible for failure recovery. Interactive editor sessions never
trigger failure recovery.
In addition, the following scenarios are not eligible for failure recovery:
- Infrastructure issues: Momentic will not recover from infrastructure
issues such as network timeouts, server errors, full-page errors, or 500
response codes. We recommend you configure retries on your test or configure a
[setup] to handle these cases.
- Legitimate failures: if all steps were executed correctly and the test
fails due to a legitimate error, failure recovery will not be triggered. This
includes cases where the UI has changed and is now out of sync with the test.
- Configuration errors: Momentic will not attempt to salvage tests that have
incorrect test data, contain illogical action sequences, or are otherwise
invalid.
