Preparing a test environment

​

Requirements

In order to automate hundreds or thousands of end-to-end tests, and run them concurrently, we need a testing environment that meets certain requirements.

​

Stability

Applications on the environment do not have frequent restarts or resource consumption issues; and there is some form of change control to prevent unexpected deployments.

​

Concurrent user capacity

We run each of your tests in a separate container, each of which acts as an individual user. The test environment (and dependent databases, email servers, etc.) must have the resources to support as many concurrent users as you have tests. If the environment is under-resourced, we can run smaller batches of tests and reduce the load, but that will increase the length of your QA cycle and slow down your release velocity.

​

Configured for automated testing

Applications in the environment need to be configured so that the tests can pass. For instance:

• If the application has feature flags, the flags are set so that tests can pass.
• There is at least one user account created for Momentic, and the ability to programmatically add more as needed to run tests concurrently.
• Sub-one minute email and SMS delivery.