Common CI setups

One Momentic test suite does three jobs. Run a subset on each pull request, the full suite on every merge, and the critical flows around each deploy. The tests are the same. The trigger, the target URL, and the size of the run differ.

Job	When it runs	What it runs	Target
Pull request check	On every pull request	Tests for the changed flows	The branch’s preview deployment
Regression pass	On every merge or on a schedule	The whole test suite	Staging or production
Deploy gate	Before or after each deploy	The critical flows	The deployed URL

Every job is one momentic run command in CI. The examples below use GitHub Actions. See GitLab CI or custom setups for other providers. Each job needs MOMENTIC_API_KEY as a secret; see GitHub Actions for how to create it. A failed run exits non-zero, so CI blocks the merge or the release. Quarantined tests are the exception and do not affect the exit code by default.

Pull request check

Run the tests against the pull request’s preview deployment, so the change runs in a real browser before review. This job starts when Vercel (or any provider that posts a GitHub deployment) marks the preview as ready. It runs only the tests that AI test selection picks from the diff:

.github/workflows/pr-tests.yml

name: PR tests

on:
  deployment_status:

jobs:
  test:
    if: github.event.deployment_status.state == 'success'
    runs-on: ubuntu-latest
    timeout-minutes: 20
    env:
      MOMENTIC_API_KEY: ${{ secrets.MOMENTIC_API_KEY }}
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - uses: actions/setup-node@v4
        with:
          node-version: 22.12.0
          cache: "npm"

      - run: npm install
      - run: npx momentic install-browsers --all

      - name: Run the tests for this diff
        run: |
          npx momentic run \
            --url-override "${{ github.event.deployment_status.environment_url }}" \
            --custom-headers \
              "x-vercel-protection-bypass=${{ secrets.VERCEL_BYPASS_SECRET }}" \
              "x-vercel-set-bypass-cookie=true" \
            --ai-select --ai-select-base origin/main \
            --reporter steps --reporter junit \
            --upload-results

      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: junit
          path: reports

What each part does:

deployment_status fires when the provider reports a deployment. The if guard runs the job only for a ready deployment, and environment_url is the preview URL.
fetch-depth: 0 gives --ai-select the full history to diff against origin/main. The deployment_status event carries no pull request base, so --ai-select-base sets it. On a pull_request event you can omit it. See AI test selection.
--url-override points every test at the preview URL. It also replaces a test’s own url.
The two --custom-headers values pass Vercel’s protection bypass. Skip them if the preview is public. See Vercel preview auth.
--reporter steps prints one line per step to the CI log. --reporter junit writes a JUnit XML file to reports/ for your CI’s test summary. See JUnit outputs.

Make the job a required status check on the branch so a failing run blocks the merge. Decide how the check treats flaky tests: quarantined tests run but do not fail the check by default. Pass --skip-quarantined to skip them, or --ignore-quarantine to count every status. See quarantine.

Add tests in the same pull request

The existing suite only catches regressions in flows it already covers. To cover what the pull request changes, ask your coding agent with the momentic-spec skill:

/momentic-spec cover the user journeys changed by this pull request

The agent writes .test.yaml files into your working tree, so the new tests land in the same review as the change and run in the same CI job.

Regression pass

Run the whole test suite on every merge to main and on a nightly schedule. This job shards the suite across four runners and merges the results into one run group in the dashboard:

.github/workflows/regression.yml

name: Regression

on:
  push:
    branches: ["main"]
  schedule:
    - cron: "0 6 * * *"

jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 30
    env:
      MOMENTIC_API_KEY: ${{ secrets.MOMENTIC_API_KEY }}
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3, 4]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22.12.0
          cache: "npm"
      - run: npm install
      - run: npx momentic install-browsers --all

      - name: Run shard ${{ matrix.shard }}
        run: |
          npx momentic run \
            --env staging \
            --shard-index ${{ matrix.shard }} \
            --shard-count 4 \
            --output-dir test-results/shard-${{ matrix.shard }}

      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: test-results-${{ matrix.shard }}
          path: test-results
          retention-days: 1

  upload:
    runs-on: ubuntu-latest
    if: always()
    needs: test
    env:
      MOMENTIC_API_KEY: ${{ secrets.MOMENTIC_API_KEY }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22.12.0
          cache: "npm"
      - run: npm install
      - uses: actions/download-artifact@v4
        with:
          path: test-results
          pattern: test-results-*
          merge-multiple: true
      - run: npx momentic results merge --output-dir test-results/merged test-results
      - run: npx momentic results upload test-results/merged

--env staging selects the staging environment from momentic.config.yaml, so the same tests run against its baseUrl and credentials. See environments.
Each shard writes to its own --output-dir. The upload job merges them so the dashboard shows one run group instead of four. See GitHub Actions.

Cover flows in order of the damage a regression causes, not in order of how hard they are to automate:

Revenue and retention paths first: sign-up, checkout, the primary action your product exists to do.
Flows that have regressed before.
Flows that exercise shared code that other features depend on.

The suite is useful only if the team trusts a failure:

Auto-heal repairs a test mid-run when the UI changed, so a moved or renamed element does not fail a test that still works.
Quarantine flaky tests instead of disabling them. They still run, so you keep the signal while you fix the test.
Reuse shared setup such as login with modules so one change updates every test that depends on it.
Read the trace and screenshots for a failure in results. Give each flow an owner, so the person who triages a failure knows the flow.

If your core flows still change every week, start with the deploy gate below and grow coverage as flows settle.

Deploy gate

Run a small set of critical flows against the new production deployment, and roll back if a flow fails. Tag the flows with a smoke label:

tests/checkout.test.yaml

fileType: momentic/test/v2
id: amber-meadow-compass
labels: [smoke, owner:payments]
steps:
  - module: ../modules/log-in.module.yaml
  - click: Checkout
  - type:
      text: 4242 4242 4242 4242
      into: Card number input
  - click: Pay now
  - assert: An order confirmation is visible

Run the labeled tests after the production deploy, and roll back when the run fails:

.github/workflows/deploy.yml

name: Deploy

on:
  push:
    branches: ["main"]

jobs:
  deploy:
    runs-on: ubuntu-latest
    outputs:
      url: ${{ steps.deploy.outputs.url }}
    steps:
      - uses: actions/checkout@v4
      - id: deploy
        run: ./scripts/deploy.sh production  # writes url=https://... to $GITHUB_OUTPUT

  smoke:
    runs-on: ubuntu-latest
    needs: deploy
    timeout-minutes: 10
    env:
      MOMENTIC_API_KEY: ${{ secrets.MOMENTIC_API_KEY }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22.12.0
          cache: "npm"
      - run: npm install
      - run: npx momentic install-browsers --all

      - name: Smoke test production
        run: |
          npx momentic run \
            --labels smoke \
            --url-override "${{ needs.deploy.outputs.url }}" \
            --retries 1 \
            --upload-results

      - name: Roll back
        if: failure()
        run: ./scripts/rollback.sh production

--labels smoke runs only tests that carry the label. A path (npx momentic run tests/smoke) or --include and --exclude also work. The include and exclude globs in momentic.config.yaml control discovery, not one run.
--retries 1 reruns a failed test once before the job fails, so one transient network error does not trigger a rollback.
Run the same job before the deploy against staging with --url-override "$STAGING_URL" if you want a gate in front of the release too.

Keep the set small so it finishes in minutes and does not grow into a second regression suite:

Flows whose failure is an incident: login, checkout, the primary create or submit action.
One happy path per flow. Leave edge cases to the regression pass.
End-to-end paths, so one test runs against the real stack.

For a flow whose steps differ by environment or feature flag, use an AI action with a goal and a postcondition instead of a fixed list of clicks:

tests/signup.test.yaml

fileType: momentic/test/v2
id: gentle-river-lantern
labels: [smoke]
steps:
  - act:
      goal: Complete the new-user signup flow using a fresh email.
      postcondition: The welcome screen is visible.

See agentic testing for when to use an AI action.

GitHub Actions

Authentication, sharding, and result upload in detail.

AI test selection

Pick the tests to run from a git diff.

Quarantine

Isolate unstable tests without blocking the rest of the suite.

Environments

Point the same tests at staging or production.

Get started

Building with AI

Author

Run

Learn

Platforms

Integrations

Advanced

Migrations

Account

Pull request check

Add tests in the same pull request

Regression pass

Deploy gate

GitHub Actions

AI test selection

Quarantine

Environments

​Pull request check

​Add tests in the same pull request

​Regression pass

​Deploy gate

​Related

GitHub Actions

AI test selection

Quarantine

Environments

Pull request check

Add tests in the same pull request

Regression pass

Deploy gate

Related