Monitoring at high scale for very heterogeneous distributed systems

As the portfolio of products of a company expands, so do the number of components to keep track of, and with that, it also highly increases the complexity of the systems in place. For product based companies it is essential to have a deep and responsive monitoring system, in order to promptly identify, respond and fix critical issues. This thesis will present a state-of-the-art monitoring system developed for a real-world, fast-growing enterprise to deal with this situation. Given the scope of The Company, largely based on acquisitions, and the volume of transactions, it is clear that the infrastructure behind all the different products will be extremely heterogeneous and time-critical, and as such, a very flexible and efficient monitoring solution will need to be implemented. This thesis will also highlight the main key strengths that such a model offers, demonstrating practical real-life scenarios and situations, comparing it with the old system that was in place in The Company as well as some other typical monitoring solutions that are commonly found in the tech industry. The analysis focuses on a several main aspects, such as cost, ease of setup and use, scalability and resilience to traffic spikes.