> ## Documentation Index > Fetch the complete documentation index at: https://momentic.ai/docs/llms.txt > Use this file to discover all available pages before exploring further. > ## Agent Instructions > The canonical docs host is https://momentic.ai/docs. Append .md to any page URL for clean Markdown. Fetch https://momentic.ai/docs/llms.txt for the page index and https://momentic.ai/docs/llms-full.txt for every page in one file. > For an unattended setup, run `npx --yes @momentic/wizard@latest -y --platform web` (use `--platform ios` or `--platform android` for mobile). Never run an interactive command in a non-TTY shell. > You cannot complete the browser sign-in yourself. Read the API key from the MOMENTIC_API_KEY environment variable, or ask the user to create one at https://app.momentic.ai/settings/api-keys and paste it back. > Momentic's capability summary for agents is at https://momentic.ai/docs/skill.md. Install the skills with `npx momentic skills --yes` (or `npx momentic-mobile skills --yes` for mobile). > Tests are YAML files in the user's repository. Use only the step names listed at https://momentic.ai/docs/reference/commands.md and the file structure at https://momentic.ai/docs/core-concepts/file-format.md. Do not invent step names, config keys, or CLI flags. > Web tests run on Chromium, iOS tests on simulators, and Android tests on emulators. Physical devices are not supported. # Common CI setups > Run the same tests as a pull request check, a full regression pass, and a deploy gate, with the CI config for each. One Momentic test suite does three jobs. Run a subset on each pull request, the full suite on every merge, and the critical flows around each deploy. The tests are the same. The trigger, the target URL, and the size of the run differ. | Job | When it runs | What it runs | Target | | ------------------ | ------------------------------- | --------------------------- | ------------------------------- | | Pull request check | On every pull request | Tests for the changed flows | The branch's preview deployment | | Regression pass | On every merge or on a schedule | The whole test suite | Staging or production | | Deploy gate | Before or after each deploy | The critical flows | The deployed URL | Every job is one `momentic run` command in CI. The examples below use GitHub Actions. See [GitLab CI](/docs/running-tests/ci/gitlab-ci) or [custom setups](/docs/running-tests/ci/custom-setups) for other providers. Each job needs `MOMENTIC_API_KEY` as a secret; see [GitHub Actions](/docs/running-tests/ci/github-actions#authentication) for how to create it. A failed run exits non-zero, so CI blocks the merge or the release. Quarantined tests are the exception and do not affect the exit code by default. ## Pull request check Run the tests against the pull request's preview deployment, so the change runs in a real browser before review. This job starts when Vercel (or any provider that posts a GitHub deployment) marks the preview as ready. It runs only the tests that AI test selection picks from the diff: ```yaml .github/workflows/pr-tests.yml theme={null} name: PR tests on: deployment_status: jobs: test: if: github.event.deployment_status.state == 'success' runs-on: ubuntu-latest timeout-minutes: 20 env: MOMENTIC_API_KEY: ${{ secrets.MOMENTIC_API_KEY }} steps: - uses: actions/checkout@v4 with: fetch-depth: 0 - uses: actions/setup-node@v4 with: node-version: 22.12.0 cache: "npm" - run: npm install - run: npx momentic install-browsers --all - name: Run the tests for this diff run: | npx momentic run \ --url-override "${{ github.event.deployment_status.environment_url }}" \ --custom-headers \ "x-vercel-protection-bypass=${{ secrets.VERCEL_BYPASS_SECRET }}" \ "x-vercel-set-bypass-cookie=true" \ --ai-select --ai-select-base origin/main \ --reporter steps --reporter junit \ --upload-results - uses: actions/upload-artifact@v4 if: always() with: name: junit path: reports ``` What each part does: * `deployment_status` fires when the provider reports a deployment. The `if` guard runs the job only for a ready deployment, and `environment_url` is the preview URL. * `fetch-depth: 0` gives `--ai-select` the full history to diff against `origin/main`. The `deployment_status` event carries no pull request base, so `--ai-select-base` sets it. On a `pull_request` event you can omit it. See [AI test selection](/docs/ai/select). * `--url-override` points every test at the preview URL. It also replaces a test's own `url`. * The two `--custom-headers` values pass Vercel's protection bypass. Skip them if the preview is public. See [Vercel preview auth](/docs/guides/auth/vercel-previews). * `--reporter steps` prints one line per step to the CI log. `--reporter junit` writes a JUnit XML file to `reports/` for your CI's test summary. See [JUnit outputs](/docs/guides/reporting/junit-outputs). Make the job a required status check on the branch so a failing run blocks the merge. Decide how the check treats flaky tests: quarantined tests run but do not fail the check by default. Pass `--skip-quarantined` to skip them, or `--ignore-quarantine` to count every status. See [quarantine](/docs/reliability/auto-maintenance#quarantine). ### Add tests in the same pull request The existing suite only catches regressions in flows it already covers. To cover what the pull request changes, ask your coding agent with the [`momentic-spec` skill](/docs/coding-agents/skills#spec-driven-development): ```text theme={null} /momentic-spec cover the user journeys changed by this pull request ``` The agent writes `.test.yaml` files into your working tree, so the new tests land in the same review as the change and run in the same CI job. ## Regression pass Run the whole test suite on every merge to `main` and on a nightly schedule. This job shards the suite across four runners and merges the results into one run group in the dashboard: ```yaml .github/workflows/regression.yml theme={null} name: Regression on: push: branches: ["main"] schedule: - cron: "0 6 * * *" jobs: test: runs-on: ubuntu-latest timeout-minutes: 30 env: MOMENTIC_API_KEY: ${{ secrets.MOMENTIC_API_KEY }} strategy: fail-fast: false matrix: shard: [1, 2, 3, 4] steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: 22.12.0 cache: "npm" - run: npm install - run: npx momentic install-browsers --all - name: Run shard ${{ matrix.shard }} run: | npx momentic run \ --env staging \ --shard-index ${{ matrix.shard }} \ --shard-count 4 \ --output-dir test-results/shard-${{ matrix.shard }} - uses: actions/upload-artifact@v4 if: always() with: name: test-results-${{ matrix.shard }} path: test-results retention-days: 1 upload: runs-on: ubuntu-latest if: always() needs: test env: MOMENTIC_API_KEY: ${{ secrets.MOMENTIC_API_KEY }} steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: 22.12.0 cache: "npm" - run: npm install - uses: actions/download-artifact@v4 with: path: test-results pattern: test-results-* merge-multiple: true - run: npx momentic results merge --output-dir test-results/merged test-results - run: npx momentic results upload test-results/merged ``` * `--env staging` selects the `staging` environment from `momentic.config.yaml`, so the same tests run against its `baseUrl` and credentials. See [environments](/docs/configuration/environments). * Each shard writes to its own `--output-dir`. The `upload` job merges them so the dashboard shows one run group instead of four. See [GitHub Actions](/docs/running-tests/ci/github-actions#sharding). Cover flows in order of the damage a regression causes, not in order of how hard they are to automate: * Revenue and retention paths first: sign-up, checkout, the primary action your product exists to do. * Flows that have regressed before. * Flows that exercise shared code that other features depend on. The suite is useful only if the team trusts a failure: * [Auto-heal](/docs/reliability/auto-maintenance#locator-auto-healing) repairs a test mid-run when the UI changed, so a moved or renamed element does not fail a test that still works. * Quarantine flaky tests instead of disabling them. They still run, so you keep the signal while you fix the test. * Reuse shared setup such as login with [modules](/docs/core-concepts/modules) so one change updates every test that depends on it. * Read the trace and screenshots for a failure in [results](/docs/running-tests/results). Give each flow an owner, so the person who triages a failure knows the flow. If your core flows still change every week, start with the deploy gate below and grow coverage as flows settle. ## Deploy gate Run a small set of critical flows against the new production deployment, and roll back if a flow fails. Tag the flows with a `smoke` label: ```yaml tests/checkout.test.yaml theme={null} fileType: momentic/test/v2 id: amber-meadow-compass labels: [smoke, owner:payments] steps: - module: ../modules/log-in.module.yaml - click: Checkout - type: text: 4242 4242 4242 4242 into: Card number input - click: Pay now - assert: An order confirmation is visible ``` Run the labeled tests after the production deploy, and roll back when the run fails: ```yaml .github/workflows/deploy.yml theme={null} name: Deploy on: push: branches: ["main"] jobs: deploy: runs-on: ubuntu-latest outputs: url: ${{ steps.deploy.outputs.url }} steps: - uses: actions/checkout@v4 - id: deploy run: ./scripts/deploy.sh production # writes url=https://... to $GITHUB_OUTPUT smoke: runs-on: ubuntu-latest needs: deploy timeout-minutes: 10 env: MOMENTIC_API_KEY: ${{ secrets.MOMENTIC_API_KEY }} steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: 22.12.0 cache: "npm" - run: npm install - run: npx momentic install-browsers --all - name: Smoke test production run: | npx momentic run \ --labels smoke \ --url-override "${{ needs.deploy.outputs.url }}" \ --retries 1 \ --upload-results - name: Roll back if: failure() run: ./scripts/rollback.sh production ``` * `--labels smoke` runs only tests that carry the label. A path (`npx momentic run tests/smoke`) or `--include` and `--exclude` also work. The `include` and `exclude` globs in `momentic.config.yaml` control discovery, not one run. * `--retries 1` reruns a failed test once before the job fails, so one transient network error does not trigger a rollback. * Run the same job before the deploy against staging with `--url-override "$STAGING_URL"` if you want a gate in front of the release too. Keep the set small so it finishes in minutes and does not grow into a second regression suite: * Flows whose failure is an incident: login, checkout, the primary create or submit action. * One happy path per flow. Leave edge cases to the regression pass. * End-to-end paths, so one test runs against the real stack. For a flow whose steps differ by environment or feature flag, use an AI action with a goal and a postcondition instead of a fixed list of clicks: ```yaml tests/signup.test.yaml theme={null} fileType: momentic/test/v2 id: gentle-river-lantern labels: [smoke] steps: - act: goal: Complete the new-user signup flow using a fresh email. postcondition: The welcome screen is visible. ``` See [agentic testing](/docs/core-concepts/agentic-testing) for when to use an AI action. ## Related Authentication, sharding, and result upload in detail. Pick the tests to run from a git diff. Isolate unstable tests without blocking the rest of the suite. Point the same tests at staging or production.