Rolling up a Google Flights search
The examples below use a flight search on Google Flights to show the core technique of the AI action step. You take a series of low-level steps whose only purpose is to reach some state, then collapse them into a single goal that names that state. The imperative version below spells out every click and keystroke, so the actual intent of searching two cities for two dates gets buried in the mechanics, and a reader has to reconstruct it from the steps.flight-search.test.yaml
flight-search.test.yaml
Assert the end state with a postcondition
The agent cannot pass its own step. When it claims success, an independent reviewer reads the settled page state, the actions the harness recorded, and your goal, then decides whether the goal’s outcome happened. You write an explicitpostcondition when the goal alone does not tell the reviewer what
“done” should look like. For a flight search you might require that results
rendered, not just that the form was submitted. A postcondition is a protected
requirement the reviewer must also confirm, and the agent cannot skip or edit
it.
flight-search.test.yaml
postcondition is the step’s contract: it spells out what “done” means for
that act. That makes each step a clean checkpoint when you chain them
together, so one act hands off to the next at a known state instead of
wherever the goal happened to stop. The
section on guards below goes
deeper and covers preconditions too.
Group goals into semantic phases
The rollup above has a flip side: do not collapse an entire test into one giant goal either. Aim for a few AI actions that each map to a phase a reader would recognize. Eachact becomes a labeled checkpoint in the trace, so when
something fails you know which phase broke.
browse-products.test.yaml
Guarding a flow with pre- and post-conditions
The rollup introducedpostcondition. Its counterpart is precondition, which
guards the start of a flow. Both are protected assertions the agent cannot
edit or skip, so they are the definition of “did this work”. A postcondition
is more than a pass/fail gate, because if it is not met when the agent finishes,
the agent treats it as a target and keeps working toward it, failing the step
only if it cannot get there. A flow that “completed” in the wrong place is
therefore either corrected or fails loudly.
login-guarded.test.yaml
precondition to fail fast and clearly when setup is wrong (wrong page,
not logged out) instead of letting the agent retry without a plan. Add an
explicit postcondition whenever the goal itself does not pin down the end
state of a state-changing flow.
Pre- and post-conditions follow the same rules as AI
check steps. Assert structural,
qualitative facts (“a confirmation is shown”), and never exact counts, prices,
timestamps, or IDs that change between runs.
Feeding the agent context it cannot infer
The agent only knows what is on screen. Anything external (an invite code, a specific record to act on, credentials) must come in through variables. Reference them with{{ env.NAME }} in
the goal.
login.test.yaml
Branching on UI that only sometimes appears
Some UI appears on only some runs: cookie banners, “what’s new” modals, A/B interstitials. When the agent resolves that UI, the cache stores anIf <condition> branch with the steps for that case, plus an optional else
branch. Later runs re-evaluate the condition against the live UI and replay only
the branch that applies, so a flow that differs between runs no longer needs the
agent every time.
The condition is a check, so it follows check rules: write the goal so the
condition can be a structural, qualitative fact (“a cookie banner is shown”).
Everything inside the branch replays from cache. If the steps in a branch no
longer match the UI, the cache busts and the step heals like any other cached
step. See the step cache.
Limitations of AI action
An AI action is a bounded planning agent, not a general program. Designing within these limits keeps tests fast and deterministic.No unbounded or infinite loops
The agent will not “keep clicking Next until the list is empty” forever. It runs a bounded plan with a retry budget and then stops, and it will not repeat an ineffective interaction more than a couple of times. If your intent is to repeat until some condition, express the bound explicitly and keep it finite.- Avoid
Keep loading more results until every order is on screen. - Prefer
Click "Load more" up to 3 times, then stop.