Scaling Stream Processing in Cloud-Native Environments

This thesis addresses the use of cloud-native autoscalers, including Horizontal Pod Autoscaler (HPA), KEDA, and Cluster Autoscaler, in conjunction with streaming processing platforms like KStream and Flink within Kubernetes environments. As Kubernetes emerges as the de-facto standard for container orchestration, the demand for effective autoscaling solutions to manage dynamic workloads has grown significantly. This study evaluates the performance and scalability of selected autoscalers trialled in real-world scenarios created with a benchmarking framework. Particular attention is paid to complex streaming use cases, such as hierarchical aggregation, which expose the inherent limitations of existing autoscaling strategies. Through an analysis of different deployment configurations, scaling techniques, and performance metrics, this work seeks to replicate results and claims from published literature, and consolidate the corresponding insights. The findings discussed in this document aim to provide practical guidance for optimizing the use of autoscalers in cloud-native architectures, enhancing both scalability and operational efficiency.