Resource management in distributed streaming applications

Distributed systems are usually implemented as networked computers communicating by means of the message passing paradigm. In recent years the need of processing huge amounts of data, rapidly and in real-time pushed the use of streaming models where software platforms are supposed to process flows of incoming data (i.e., streams). This “Data in motion” approach is opposed to the more traditional “batch approach” where the velocity and the real-time dimensions are not important. We can see streaming applications in action in many fields such as: system monitoring, Internet of Things (IoT), social networks, etc. In this thesis we are going to describe the main frameworks for streaming management, compare them according to their architectures, performances and how they implement some advanced real-time applications (e.g., Machine Learning workloads). Finally, we propose a framework for resource management in distributed streaming applications.