Locating elements

​

Writing element descriptors

​

Include accessibility attributes

Momentic’s AI is primarily trained to understand the web accessibility tree, which is how screen readers and other assistive technologies navigate web pages. As such, the best descriptions reference elements using their accessibility attributes.

Most importantly, we recommend mentioning the desired element’s web accessibility role in the description, such as “the submit button”, “the homepage link”, “the password input”, “the ‘People’ tab”, or “the combobox for selecting a month”.

Other accessibility attributes are also helpful to include, such as aria-labels, alt text, placeholder text, and graphic titles. Never include auto-generated or lengthy attributes that may change over time, such as full URLs or UUIDs.

Momentic’s AI is trained to tolerate minor inaccuracies in descriptions, but if you are unsure what attributes to use, you can use Chrome Dev Tools to find out. Note that if your desired element has no accessibility attributes, Chrome may show it as an “ignored” or “generic” element:

​

Include HTML attributes and content

Momentic also passes certain high-signal HTML attributes to its AI agents. These attributes can be customized for Enterprise customers, but by default include the list below. Note that attribute values that are automatically flagged as auto-generated, sensitive, or lengthy are not passed to the AI.

id
value
type
class
height
width
target
title
href
src
alt
role
headers
scope
checked
required
action

min
max
minlength
maxlength
multiple
pattern
placeholder
accept
contenteditable

data-value
data-testid
data-cy
data-test-id
data-test
data-role
data-type
data-key
data-action
data-hidden

data-handleid
data-handlepos

data-col-index
data-row-index
data-row
data-col

If the desired element contains text content, you can also include a subset or summary of the text in your description. For example, the “the ‘Welcome’ header” or “the text area containing ‘Lorem Ipsum’…”.

​

Include visual attributes

Momentic’s AI is also trained to understand visual attributes like colors, shapes, sizes, and positions. For example, you can say “the red ‘Cancel’ button”, “the thumbs up icon”, or “the hamburger menu beside the logo”.

Due to image quality constraints, we recommend against relying on visual attributes alone to identify elements smaller than 12x12 pixels on the page. Visual-based descriptions are also less reliable when the desired element is embedded among many other elements with similar sizes and features (e.g. a toolbar full of small icons). In these cases, we recommend combining the visual attributes with other cues.

​

Include positional cues

In general, Momentic’s AI does an excellent job identifying elements by its name or the text it contains as it appears on the screen; however, in some cases (e.g., when there are multiple instances of the same element on the screen), you will need to specify a relative location to provide a better context to identify it.

You can provide cues such as “…below the ‘Introduction’ header” or “…beside the ‘Back’ button”. Only use positioning cues if they are expected to be stable over time.

You can also use the inside keyword to specify the parent element of another element. For example, “the ‘Submit’ button inside the user dialog”.

You can also use the nth keyword to specify the position of an element in a list. For example, “the second ‘Submit’ button”.

You can also use the closest keyword to specify the closest element to another element. For example, “the closest ‘Submit’ button to the ‘Username’ input”.

​

Use quotations for exact matches

Momentic’s AI is trained to treat single quotation marks as a request a for a nearly exact match (small typos and capitalization differences are still tolerated). As such, the description “the ‘Submit’ button” will never match a button with the text ‘Approve’.

If quotation marks are not provided, Momentic’s AI may try to use the context available on the page to infer the correct element if an exact match is not found. This enables auto-healing in the event of small changes to the page.

​

Prohibit other choices

If there are multiple elements on the page with similar attributes, you can explicitly prohibit Momentic’s AI from choosing certain options. For example: “The ‘Submit’ button inside the confirm modal. Do not choose any buttons in the form.” However, this method is less effective than providing the desired element with a stable, unique attribute like an aria-label.

​

Last resorts

​

Using data-testids and CSS selectors

Momentic supports CSS selectors as an escape hatch. We don’t recommend this as it is much more brittle and can lead to test flakiness. You can turn on CSS selector support by toggling the option in the step settings.

You are able to locate elements using data-testid attributes. For example: [data-testid="value"].

You are also able to locate elements by using CSS selectors. For example: input#username or button.mui.primary.

​

Using X/Y coordinates

Momentic supports interacting with the page using X / Y coordinates, specified in device-independent CSS pixels. This can be useful for interacting with inaccessible elements such as canvases or for clicking at a particular location on the screen that does not necessary correspond to an element (e.g. to dismiss a modal).

Note that using X / Y coordinates is less maintainable than using element descriptors, as the position of elements on the page can change over time. In addition, websites may render differently on different devices and operating systems, making X / Y coordinates less portable. To determine or debug X / Y coordinates, we recommend running this JavaScript function in the Chrome DevTools console.

​

Common pitfalls

​

Dynamic descriptions

Momentic caches AI locator results based on your element description. If you use dynamic terms like nth or closest in a description, make sure to turn caching off for that step. This will ensure that Momentic will always re-evaluate the description on every test run instead of attempting to reuse cached information. This may impact credit usage.

When using template strings (e.g. {{ env.ROW_NAME }}) in an element description, the cache key includes the resolved value of the template string. As such, if the value of the template string changes between test runs for a given step, the cache will be invalidated and the element will be re-located.

​

Hidden elements

Momentic’s AI will ignore elements that are explicitly marked as aria-hidden (or are part of sub-trees that are hidden), not attached to the DOM, or have no bounding box.