Monte Carlo Methods for Hyperparameters Handling in Bayesian Optimization for IT Configuration Autotuning

Configuring an IT system is essential to increase its performance and reduce costs. However, the number of available parameters and their interdependencies make system configuration a very complex problem. Bayesian Optimisation (BO) is a powerful tool for the minimisation of cost functions that are expensive to evaluate and thus finds valid application in the automatic optimisation of such systems. In particular, cost functions associated to systems' performances may exhibit spatial non-stationarity, a challenging complication that has recently been approached with the Beta warping strategy. However, the practical application of this technique requires selecting different hyper-parameters that control the behaviour of the optimiser and are too costly to test extensively. One possible approach is to treat the hyper-parameters of both the optimiser and Beta warping in an entirely Bayesian manner. That is, instead of selecting a single value, several possible values, weighted by their probability, are considered to compute a single acquisition function. Nonetheless, the choice of the set of hyper-parameter values on which to evaluate the acquisition function is again crucial for the optimisation result. In this thesis, therefore, different Markov Chain Monte Carlo (MCMC) sampling approaches for handling hyper-parameters in Bayesian Optimisation are evaluated, combined or not with the Beta warping strategy. The following strategies were implemented: maximum likelihood point-estimate as a baseline, marginalisation of the acquisition function on MCMC extracted samples with Lognormal distribution, Gaussian jump distribution, Gaussian jump distribution with tuning of the covariance matrix using the Vihola algorithm and EMCEE sampler. The implementations were tested first by applying them to analytical benchmark functions and then on real datasets, showing a trend of improved performance resulting from the use of the EMCEE sampling strategy in conjunction with the application of Beta warping.