Automated Performance Testing in Ephemeral Environments

Conducting performance tests on applications is a crucial step in the software development process. Many companies have dedicated teams responsible for monitoring application performance (APM) and conducting necessary tests on developed components, services and applications. However, due to its distinct nature compared to other common tests, performance testing is not frequently carried out on a daily basis within companies. Nevertheless, it is essential to integrate continuous performance testing during and after feature development to ensure that the system under test meets the desired perfor- mance metrics. In this thesis, we discuss the existing approach in a static en- vironment and its limitations, then propose a new approach in an ephemeral environment, and perform common types of performance testing including load tests and stress tests. Additionally, as the needs of the application evolve, the design of the system changes accordingly. Therefore we need a way to apply and evaluate these changes as a part of the testing process. This is where ephemeral environ- ments help. Using ephemeral environments, rather than static ones, enables us to configure and deploy the required infrastructure within each run, allow- ing us to match the same production setup or quickly assess the performance of different alternatives. This improves reliability, flexibility as well as re- peatability of the tests, helping to properly automate them instead of relying on manual actions. Moreover, this simplified setup and execution of perfor- mance tests - entirely handled as code - enables software engineers to incor- porate testing into their daily routines, reducing the reliance on centralized APM teams. The conclusion of this thesis includes developing an automated performance test application, leveraging ephemeral environments to facilitate the testing process. A temporary environment, configured and scaled very much like the production one, is created and used to run performance tests, then destroyed at the end of the run to mitigate cloud-related costs.