Prompting 101
Writing assertions
Guide to asserting on complex application state with Momentic’s AI
Momentic’s AI is trained to understand HTML, the accessibility tree, and screenshots. Use a combination of these three modalities to verify assertions. Read more here.
Best practices
- Keep assertions short and specific. Break down complex assertions into smaller, more manageable parts. This helps the AI understand the context and reduces the chance of misinterpretation. We recommend using no more than 20 words.
- Provide useful context. To avoid misinterpretations, include information such as a specific part of the page the AI should focus on or the element you are looking for.
- Avoid jargon. The AI can be confused by non-standard terminology.
Examples
Here are some examples of how to write assertions using AI check:
| Category | Example |
|---|---|
| Presence (or absence) of text | Verify that a congratulations message is on the page |
| Presence (or absence) of elements | Verify that the page shows 3 search results |
| Logical statements | The publish date of this article is more than 30 days ago |
| Page state | There is no error message present on the page |
| Color | Verify the buy button is blue |
| Layout | The ‘XL’ button is the largest button on the page |
| Images & video | There is a LinkedIn icon on the page |
Data visualizations
You can also assert on SVG or canvas-based data visualizations rendered by libraries such as D3, Hicharts, Recharts, and more.
To improve accuracy, use the following guidelines:
- Maximize or expand your graph so that it takes up a majority of the screen. If your charting library does not have this functionality, you can try setting
document.body.style.zoommanually using JavaScript. - Provide clear labels and units for all axes. Include grid lines when possible.
- For interactive visualizations, use Click or Hover steps to show tooltips and labels before making assertions.
Troubleshooting
- Assertions with actions: If your assertion requires an action (e.g., “expand the dropdown”), the AI may not be able to understand it. Instead, split the action and assertion into separate steps.
- Assertions over time: The AI does not support assertions that depend on time (e.g., “after 5 seconds, a popup appears”). Instead, use a Wait step to pause the test and then check for the expected condition.
- Assertions on small visual details: Due to resolution downscaling performed by AI providers, the AI may not be able to perceive small visual details such as exact hex codes, colors of indicator dots, colors of thin font faces, and shapes of SVGs smaller than 12px. We recommend keeping assertions general enough to avoid these edge cases.
- Complex math: The AI may struggle with complex mathematical calculations (e.g., “the average slope of the graph is 31.4”). Instead, simplify the assertion to a more straightforward comparison (e.g., “the graph shows an upward trend”).
- Complex conditionals: The AI may not handle complex conditional logic well (e.g., “if the user is logged in, check that their profile photo is visible; otherwise, check that there is a Google icon on the page”). Instead, break it down into simpler assertions that do not rely on conditional logic.