Smoke test preview deploys

​

The tests

fileType: momentic/test/v2
id: signup-smoke
labels: [smoke]
steps:
  - navigate: /signup
  - act:
      goal: Complete the new-user signup flow using a fresh email
      postcondition: The welcome screen is visible

​

The workflow

name: Preview smoke

on:
  deployment_status:

jobs:
  test:
    if: >-
      github.event.deployment_status.state == 'success' &&
      github.event.deployment.environment != 'Production'
    runs-on: ubuntu-latest
    timeout-minutes: 15
    env:
      MOMENTIC_API_KEY: ${{ secrets.MOMENTIC_API_KEY }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22.12.0
      - run: npm install
      - run: npx momentic install-browsers chromium

      - name: Smoke test the preview
        run: |
          npx momentic run --labels smoke -y --upload-results \
            --url-override "${{ github.event.deployment_status.environment_url }}" \
            --custom-headers \
              "x-vercel-protection-bypass=${{ secrets.VERCEL_AUTOMATION_BYPASS_SECRET }}" \
              "x-vercel-set-bypass-cookie=true"

• deployment_status fires for Vercel and any provider that posts a GitHub deployment. environment_url is the preview URL. For providers that do not post deployments, poll for the URL first: see the Next.js on Vercel guide for the wait-for-preview variant.
• deployment.environment != 'Production' keeps staging and production deployments from re-running the preview suite. Match the environment names your provider posts (Vercel uses Preview and Production).
• --url-override replaces each test’s url and each environment’s baseUrl.
• Drop the --custom-headers pair when the preview is public. For protection-bypass details, see Vercel preview auth.

​

Decisions

• Labels vs selection. --labels smoke is a fixed list; --ai-select picks tests from the diff. Use the label for a fast universal gate, selection when the suite is large, or both.
• What belongs in smoke. A handful of end-to-end happy paths that finish in minutes. If the preview job takes longer than a reviewer waits, it is doing regression work. Move those tests to the nightly suite.
• Failure is not rollback. A preview failure blocks a merge; it does not roll anything back. For post-deploy rollback gating, see the deploy-gate workflow in Common CI setups.

​

Related

• Common CI setups
• Next.js on Vercel
• Vercel preview auth
• Gate pull requests on critical flows