Prompting 101
Writing assertions
Guide to asserting on complex application state with Momentic’s AI
Momentic’s AI is trained to understand HTML, the accessibility tree, and screenshots. Use a combination of these three modalities to verify assertions. Read more here.
Best practices
- Keep assertions short and specific. Break down complex assertions into smaller, more manageable parts. This helps the AI understand the context and reduces the chance of misinterpretation.
- Provide useful context. To avoid misinterpretations, include information such as a specific part of the page the AI should focus on or the element you are looking for.
- Avoid jargon. The AI can be confused by non-standard terminology.
Examples
Here are some examples of how to write assertions using AI check:
| Category | Example |
|---|---|
| Presence (or absence) of text | Verify that a congratulations message is on the page |
| Presence (or absence) of elements | Verify that the page shows 3 search results |
| Logical statements | The publish date of this article is more than 30 days ago |
| Page state | There is no error message present on the page |
| Color | Verify the buy button is blue |
| Layout | The ‘XL’ button is the largest button on the page |
| Images & video | There is a LinkedIn icon on the page |
Data visualizations
You can also assert on SVG or canvas-based data visualizations rendered by libraries such as D3, Hicharts, Recharts, and more.
To improve accuracy, use the following guidelines:
- Maximize or expand your graph so that it takes up a majority of the screen. If your charting library does not have this functionality, you can try setting
document.body.style.zoommanually using JavaScript. - Provide clear labels and units for all axes. Include grid lines when possible.
- For interactive visualizations, use Click or Hover steps to show tooltips and labels before making assertions.
Troubleshooting
- Assertions with actions: If your assertion requires an action (e.g., “expand the dropdown”), the AI may not be able to understand it. Instead, split the action and assertion into separate steps.
- Assertions over time: The AI does not support assertions that depend on time (e.g., “after 5 seconds, a popup appears”). Instead, use a Wait step to pause the test and then check for the expected condition.
- Complex math: The AI may struggle with complex mathematical calculations (e.g., “the average slope of the graph is 31.4”). Instead, simplify the assertion to a more straightforward comparison (e.g., “the graph shows an upward trend”).
- Complex conditionals: The AI may not handle complex conditional logic well (e.g., “if the user is logged in, check that their profile photo is visible; otherwise, check that there is a Google icon on the page”). Instead, break it down into simpler assertions that do not rely on conditional logic.