Best practices
- Keep assertions short and specific. Aim for under 20 words; split compound assertions into multiple steps.
- Provide context. Point at the relevant region of the page and define any non-standard terminology.
- Remove ambiguity. The answer should be clearly true or false given the page state. When in doubt, use single quotes for exact text matches.
Good assertions
| Category | Example |
|---|---|
| Presence or absence of text | A congratulations message is shown in the modal |
| Presence or absence of elements | The page shows 3 search results |
| Logical statements | The publish date of this article is more than 30 days ago |
| Page state | There is no error message present on the page |
| Color | The buy button is blue |
| Layout | The ‘XL’ button is the largest button in the submission form |
| Images and video | There is a LinkedIn icon on the page |
Bad assertions
| Problem | Bad | Better |
|---|---|---|
| Malformed | Search results | There are search results displayed on the main body |
| Ambiguous | The order of the results is correct | The search results are sorted by date |
| Multiple focuses | The logo is blue, the page shows 4 results, and the last result is more than 30 days old | Split into 3 separate assertions |
| Contains an action | Wait for the results to appear | Use a Wait step, then assert the results appeared |
| Imperceivable detail | The color of line divider is #ff5733 | The color of the divider looks red |
| Subjective | The page looks well designed | No text in the table overflows its cell |
| Time-sensitive | The video plays a 10-second advertisement | Assert on DOM state instead |
Pre-trained behavior
Typos
The agent tolerates minor typos.john@@gmail.com will match [email protected].
For exact matching, wrap the expected value in single quotes and instruct the
agent explicitly:
the username is exactly 'john@@gmail.com'. do not allow any typos.
When generating test data programmatically, prefer long, distinct values to
avoid accidental matches.