Skip to main content

Documentation Index

Fetch the complete documentation index at: https://momentic.ai/docs/llms.txt

Use this file to discover all available pages before exploring further.

AI-powered features for test runs.

​
classify

Use an AI agent to classify the result of a given test run. The agent inspects run data (step results, screenshots, and error details) and returns a category and reasoning for the result.
Currently, only failed runs are classified. Passing runs will return NO_FAILURE.
npx momentic ai classify [options] <runIdOrUrl>

​
Arguments

​
<runIdOrUrl>
string
required
The ID of the run to classify, or the full run URL (e.g. https://app.momentic.ai/runs/<runId>). The run must be either uploaded to Momentic dashboard or in the latest run group run by the CLI, stored locally under the project output directory.

​
Categories

The classification result will be one of the following categories:
​
NO_FAILURE
enum
The run had no failures; all attempts passed.
​
RELATED_APPLICATION_CHANGE
enum
The test is out of date with an intentional change in the app flow or UI. The test needs to be updated to reflect the new behavior.
​
RELATED_APPLICATION_BUG
enum
A failure tied to the test’s intended behavior where the app is clearly broken, not changed (e.g. an unexpected error message appeared, or something that was expected to happen clearly did not).
​
UNRELATED_APPLICATION_CHANGE
enum
The test is out of date with an intentional change in the app, but the change is unrelated to what the test is specifically verifying.
​
UNRELATED_APPLICATION_BUG
enum
A failure unrelated to the test’s intended behavior where the app is clearly behaving incorrectly (e.g. a bug during login or navigation that prevents the test from reaching its core assertions).
​
TEST_CAN_BE_IMPROVED
enum
The test itself should be changed to prevent the failure. Common causes include race conditions or overly vague AI descriptions.
​
INFRA
enum
A rare or intermittent failure caused by outside factors (e.g. a browser crash, high resource usage).
​
PERFORMANCE
enum
Application performance issues unrelated to infrastructure (e.g. a loading spinner that never disappears, slow page transitions, an AI assertion step that timed out waiting for the page).
​
MOMENTIC_ISSUE
enum
An issue with Momentic’s own test execution (e.g. incorrect cache entries, unexpected locator redirect behavior, obvious AI hallucinations).

​
Output

By default, output is printed as plain text: 
Classification: RELATED_APPLICATION_BUG
Reasoning: The checkout button was disabled due to a validation error that appeared after the address step...
Confidence: high
Recoverable: RECOVERABLE

​
Options

​
--output-format <format>
string
Default: textHow to print the classification result. Use json for machine-readable output:
{
  "category": "RELATED_APPLICATION_BUG",
  "reasoning": "The checkout button was disabled due to a validation error...",
  "recoverable": "RECOVERABLE",
  "confidence": "high"
}
In json mode, all non-error output is suppressed so stdout contains only the final JSON payload or errors that prevent classification.
​
--quiet
boolean
Suppress the agent’s intermediate reasoning from stdout. By default, reasoning is streamed as the agent works. This flag is redundant when --output-format json is set, since JSON mode already suppresses streamed reasoning.
​
--save
boolean
Persist the classification result into the run’s metadata. After saving, downstream tooling sees this classification as if it had been produced during the run.
This is only available for classifications on local run zips currently.