Skip to main content
AI-powered features for test runs.

​
classify

Use an AI agent to classify the result of a given test run. The agent inspects run data (step results, screenshots, and error details) and returns a category and reasoning for the result.
Currently, only failed runs are classified. Passing runs will return NO_FAILURE.
npx momentic ai classify [options] <runId>
  • <runId>: The ID of the run to classify. The run must be either uploaded to Momentic Cloud or in the latest run group run by the CLI, stored locally under the project output directory.

​
Categories

The classification result will be one of the following categories:
CategoryDescription
NO_FAILUREThe run had no failures; all attempts passed.
RELATED_APPLICATION_CHANGEThe test is out of date with an intentional change in the app flow or UI. The test needs to be updated to reflect the new behavior.
RELATED_APPLICATION_BUGA failure tied to the test’s intended behavior where the app is clearly broken, not changed (e.g. an unexpected error message appeared, or something that was expected to happen clearly did not).
UNRELATED_APPLICATION_CHANGEThe test is out of date with an intentional change in the app, but the change is unrelated to what the test is specifically verifying.
UNRELATED_APPLICATION_BUGA failure unrelated to the test’s intended behavior where the app is clearly behaving incorrectly (e.g. a bug during login or navigation that prevents the test from reaching its core assertions).
TEST_CAN_BE_IMPROVEDThe test itself should be changed to prevent the failure. Common causes include race condition or overly vague AI descriptions.
INFRAA rare or intermittent failure caused by outside factors (e.g. a browser crash, high resource usage).
PERFORMANCEApplication performance issues unrelated to infrastructure (e.g. a loading spinner that never disappears, slow page transitions, an AI assertion step that timed out waiting for the page).
MOMENTIC_ISSUEAn issue with Momentic’s own test execution (e.g. incorrect cache entries, unexpected locator redirect behavior, obvious AI hallucinations).

​
Output

By default, output is printed as plain text: 
Classification: RELATED_APPLICATION_BUG
Reasoning: The checkout button was disabled due to a validation error that appeared after the address step...

​
Options

​
--output-format <format>

Default: text How to print the classification result. Use json for machine-readable output: 
{
  "category": "RELATED_APPLICATION_BUG",
  "reasoning": "The checkout button was disabled due to a validation error..."
}

npx momentic ai classify --output-format json <runId>

​
--quiet

Suppress the agent’s intermediate reasoning from stdout. By default, reasoning is streamed as the agent works. Use together with --output-format json for machine-readable output when integrating with upstream agents or systems. 
npx momentic ai classify --quiet <runId>