--- title: "How to test pull requests from coding agents on preview deploys before you review them" description: "Run end-to-end tests against the Vercel or Netlify preview of every Claude Code, Codex, Devin or Copilot pull request before a human reviews it." canonical: "https://momentic.ai/blog/test-coding-agent-pull-requests-preview-deploys" last-updated: "2026-11-02T19:11:43Z" --- # How to test pull requests from coding agents on preview deploys before you review them URL: https://momentic.ai/blog/test-coding-agent-pull-requests-preview-deploys [blog](/blog) [/ resources](/blog/category/resources) / test-coding-agent-pull-requests-preview-deploys Resources Run end-to-end tests against the Vercel or Netlify preview of every Claude Code, Codex, Devin or Copilot pull request before a human reviews it. Wei-Wei Wu CEO, Momentic ## TL;DR - For agent-authored PRs, test the deployed preview URL before reviewing the diff. A code review cannot show whether the changed flow works in a browser. - Run end-to-end tests in [GitHub Actions](/docs/running-tests/ci/github-actions) against each PR’s Vercel or Netlify preview URL. The test target should be the deployed build, not localhost. - Give the coding agent a way to run the test suite before it commits or opens the PR. The agent can inspect failures and revise its changes before CI runs. - When a PR changes behavior that no test covers, run an exploratory pass with [Mo](/mo) against the preview and review its findings alongside the PR. ## Why code review alone doesn't catch what agent PRs break Code review alone is not enough for agent-generated code. A reviewer can inspect a diff for intent, correctness, and maintainability, but the diff does not show how the deployed application behaves with real browser state or environment settings. An agent might change a shared sign-in handler and update the tests beside it. The handler may look correct while a checkout flow loses the user’s session after a redirect. A reviewer who focuses on the changed files may never open checkout, and a test confined to the sign-in page will not exercise that path. As coding agents produce more changes, reviewers have more behavior to verify. Generating a plausible patch does not establish that unaffected pages still work. Diff review remains necessary, but approval of an agent-written PR also needs evidence from the running application, especially for flows that cross pages or depend on deployed configuration. ## Run end-to-end tests against the preview URL in CI An agent-written PR needs a browser check against the code reviewers will actually open. A local test run can pass while the Vercel or Netlify preview fails because of deploy configuration, missing environment variables, or a route that behaves differently after deployment. Run the critical end-to-end flows against the PR’s preview URL after the deploy succeeds. A GitHub Actions job can trigger on `pull_request`, look up a successful preview deployment for the PR’s head commit, and pass its URL to the test suite. For example, if your host records previews as GitHub deployments, a `github-script` step can resolve the URL before Momentic runs. `on: pull_request permissions: contents: read deployments: read jobs: preview-e2e: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - id: preview uses: actions/github-script@v7 with: script: | const { owner, repo } = context.repo; const sha = context.payload.pull_request.head.sha; for (let attempt = 0; attempt < 30; attempt++) { const { data: deployments } = await github.rest.repos.listDeployments({ owner, repo, sha, per_page: 100 }); for (const deployment of deployments) { if (deployment.environment !== 'Preview') continue; const { data: statuses } = await github.rest.repos.listDeploymentStatuses({ owner, repo, deployment_id: deployment.id, per_page: 1 }); const status = statuses[0]; if (status?.state === 'success' && status.environment_url) { core.setOutput('url', status.environment_url); return; } } await new Promise(resolve => setTimeout(resolve, 10000)); } core.setFailed('No successful preview deployment for the PR commit'); - run: npx momentic run --labels critical --reporter steps --upload-results env: PREVIEW_URL: ${{ steps.preview.outputs.url }}` Configure the test suite to use `PREVIEW_URL` as its application URL. The environment variable in the workflow does not, by itself, change where a test navigates. Match the `Preview` deployment filter to the environment name your host uses, and keep the commit check so CI cannot accidentally test an older preview. If Vercel or Netlify posts the URL in a PR comment instead of a GitHub deployment status, replace the lookup step with a resolver for that comment. The `critical` label keeps the PR gate focused on flows that must pass before review or merge. The steps reporter makes failures readable in CI, while uploaded results give the reviewer evidence to inspect. [Claude Code](/blog/claude-code-for-end-to-end-testing), Codex, and Devin PRs use the same gate as human-written PRs. Agent authorship changes the need for independent verification, not the mechanics of targeting a preview. ## Make the agent run tests before it opens the PR [Codex](/blog/codex-ui-testing-guide) can catch a broken flow before it commits if it can run the application and its end-to-end tests in its workspace. Give the agent a test-running path, then instruct it to run the tests after changing code, inspect failures, fix the cause, and rerun the affected tests before committing. Install the Momentic skill in the workspace where Codex works. `npx momentic skills --yes` The Momentic MCP server provides another way to give a coding agent access to testing tools. Whichever integration you use, give the agent access to the application and the existing YAML tests. The agent should run those tests against its local instance and report what passed or failed with the PR. A passing run should refer to the final code, not an earlier revision the agent subsequently changed. Keep this pre-commit run separate from the GitHub Actions check. The agent tests the code it intends to submit and can iterate while it still has the local environment available. CI runs the committed revision against the Vercel or Netlify preview deploy, where deployment configuration or runtime behavior may differ. If the agent cannot start the app or run the tests in its workspace, record that limitation rather than treating an unrun test suite as a pass. ## Exploratory testing on the preview when there's no test yet An agent-written PR can change a flow that no existing end-to-end test covers. For example, a new checkout step may render on the preview deploy while blocking users who return with an existing cart. A reviewer skimming the diff may miss the runtime behavior, and a fixed test suite cannot check a path nobody has added to it. Use Mo for an exploratory pass against the live PR preview. Install the GitHub App, then start the documented PR flow. `qa start --pr` The flow captures the PR revision and its preview URL, so the findings refer to a specific version of the code rather than whichever deploy happens to be live later. Give Mo a brief that names the changed behavior and the paths worth trying. Specify how to sign in, which account or test data it may use, and what actions are out of bounds, such as sending real emails or placing orders. Read Mo’s findings alongside the preview and the diff. Evidence from an attempted flow can expose behavior the agent did not test and the reviewer did not think to inspect. Exploratory findings still need engineering judgment. Confirm a reported failure, decide whether it belongs to this PR, and add a repeatable test when the behavior should remain covered on later commits. ## Putting it together: a verification loop for agent PRs An agent-authored PR needs three checks because each catches a different failure mode. Before committing, the agent runs the existing tests locally and uses failures to revise its code. That catches known regressions before the reviewer sees the PR, but it cannot verify the deployed artifact. Once the PR has a preview URL, CI runs the critical end-to-end tests against that Vercel or Netlify deploy. A [required check](/blog/ci-e2e-test-gating) can block merge if the tests fail, while `--upload-results` gives the reviewer evidence to inspect. The preview run can catch differences between the agent’s local environment and the code that actually deployed. Changes without tests need a separate exploratory pass. Give Mo the PR preview and a brief describing the changed behavior and relevant boundaries. If Mo finds a failure, its report gives the agent or reviewer repro steps to investigate. You can then add a test for confirmed behavior before a later PR depends on it. Keep the failure path connected to the author of the change. The agent fixes local failures before committing, CI returns deployed failures to the PR, and exploratory findings return to the agent or reviewer for confirmation. The reviewer still decides whether the evidence covers the behavior the PR was meant to change. ## Engineering takeaway As coding agents write more of a PR, the preview deploy becomes the thing worth testing. A diff lets you inspect the code, but the preview lets you check whether the committed changes work in the deployed app. Human review still matters for intent and maintainability. To get started with Momentic, run `npx @momentic/wizard@latest`. ## Frequently asked questions Do I need physical devices? No. Mobile testing in this workflow uses simulators or emulators only. Testing on physical devices needs separate handling. How do I secure the preview URL? Keep the Vercel or Netlify preview behind your existing access controls, and give the CI runner and Mo access through authentication or allowlisting that your deployment supports. Confirm both can reach the preview before treating an access failure as a test failure. When should I use YAML tests instead of Mo? Use the YAML test framework for known flows that should run on every relevant PR. Use Mo to explore a preview when the agent changed behavior that the existing tests do not cover. Can I use this workflow for mobile PRs? Yes, if the PR produces a build you can test in a simulator or emulator. A Vercel or Netlify preview URL verifies a web deploy, not a native mobile build. Still have additional questions? ## Keep reading. [Resources Keeping end-to-end tests as YAML in your repo: review, lint and run them in CI Keep end-to-end tests as YAML next to the code: monorepo layout, reviewing test changes in pull requests, linting, and running them in CI. Wei-Wei Wu 8 min read](/blog/end-to-end-tests-yaml-in-repo)[Resources Testing AI-Generated Code: A Guide for Engineering Teams Testing AI generated code effectively requires a rethink of your software testing processes. Here’s how to prepare for the uptick in development velocity. Wei-Wei Wu 9 min read](/blog/testing-ai-generated-code)[Resources Why Claude Code Is Not Enough for End-to-End Testing Using Claude Code for end-to-end testing will make test creation faster, but you’ll need to pair it with other tools for the strongest efficiency gains. Wei-Wei Wu 8 min read](/blog/claude-code-for-end-to-end-testing) ## Close the feedback loop. Point Momentic at your app. Free to start, no credit card. [Try for free](https://app.momentic.ai/signup) [Contact sales](/sales)