Configuring an IT system is essential to increase its performance and reduce costs. However, the number of available parameters and their interdependencies make system configuration a very complex problem. Bayesian Optimisation (BO) is a powerful tool for the minimisation of cost functions that are expensive to evaluate and thus finds valid application in the automatic optimisation of such systems. In particular, cost functions associated to systems' performances may exhibit spatial nonstationarity, a challenging complication that has recently been approached with the Beta warping strategy. However, the practical application of this technique requires selecting different hyperparameters that control the behaviour of the optimiser and are too costly to test extensively. One possible approach is to treat the hyperparameters of both the optimiser and Beta warping in an entirely Bayesian manner. That is, instead of selecting a single value, several possible values, weighted by their probability, are considered to compute a single acquisition function. Nonetheless, the choice of the set of hyperparameter values on which to evaluate the acquisition function is again crucial for the optimisation result. In this thesis, therefore, different Markov Chain Monte Carlo (MCMC) sampling approaches for handling hyperparameters in Bayesian Optimisation are evaluated, combined or not with the Beta warping strategy. The following strategies were implemented: maximum likelihood pointestimate as a baseline, marginalisation of the acquisition function on MCMC extracted samples with Lognormal distribution, Gaussian jump distribution, Gaussian jump distribution with tuning of the covariance matrix using the Vihola algorithm and EMCEE sampler. The implementations were tested first by applying them to analytical benchmark functions and then on real datasets, showing a trend of improved performance resulting from the use of the EMCEE sampling strategy in conjunction with the application of Beta warping.
Configuring an IT system is essential to increase its performance and reduce costs. However, the number of available parameters and their interdependencies make system configuration a very complex problem. Bayesian Optimisation (BO) is a powerful tool for the minimisation of cost functions that are expensive to evaluate and thus finds valid application in the automatic optimisation of such systems. In particular, cost functions associated to systems' performances may exhibit spatial nonstationarity, a challenging complication that has recently been approached with the Beta warping strategy. However, the practical application of this technique requires selecting different hyperparameters that control the behaviour of the optimiser and are too costly to test extensively. One possible approach is to treat the hyperparameters of both the optimiser and Beta warping in an entirely Bayesian manner. That is, instead of selecting a single value, several possible values, weighted by their probability, are considered to compute a single acquisition function. Nonetheless, the choice of the set of hyperparameter values on which to evaluate the acquisition function is again crucial for the optimisation result. In this thesis, therefore, different Markov Chain Monte Carlo (MCMC) sampling approaches for handling hyperparameters in Bayesian Optimisation are evaluated, combined or not with the Beta warping strategy. The following strategies were implemented: maximum likelihood pointestimate as a baseline, marginalisation of the acquisition function on MCMC extracted samples with Lognormal distribution, Gaussian jump distribution, Gaussian jump distribution with tuning of the covariance matrix using the Vihola algorithm and EMCEE sampler. The implementations were tested first by applying them to analytical benchmark functions and then on real datasets, showing a trend of improved performance resulting from the use of the EMCEE sampling strategy in conjunction with the application of Beta warping.
Monte Carlo Methods for Hyperparameters Handling in Bayesian Optimization for IT Configuration Autotuning
MANARA, NOEMI
2022/2023
Abstract
Configuring an IT system is essential to increase its performance and reduce costs. However, the number of available parameters and their interdependencies make system configuration a very complex problem. Bayesian Optimisation (BO) is a powerful tool for the minimisation of cost functions that are expensive to evaluate and thus finds valid application in the automatic optimisation of such systems. In particular, cost functions associated to systems' performances may exhibit spatial nonstationarity, a challenging complication that has recently been approached with the Beta warping strategy. However, the practical application of this technique requires selecting different hyperparameters that control the behaviour of the optimiser and are too costly to test extensively. One possible approach is to treat the hyperparameters of both the optimiser and Beta warping in an entirely Bayesian manner. That is, instead of selecting a single value, several possible values, weighted by their probability, are considered to compute a single acquisition function. Nonetheless, the choice of the set of hyperparameter values on which to evaluate the acquisition function is again crucial for the optimisation result. In this thesis, therefore, different Markov Chain Monte Carlo (MCMC) sampling approaches for handling hyperparameters in Bayesian Optimisation are evaluated, combined or not with the Beta warping strategy. The following strategies were implemented: maximum likelihood pointestimate as a baseline, marginalisation of the acquisition function on MCMC extracted samples with Lognormal distribution, Gaussian jump distribution, Gaussian jump distribution with tuning of the covariance matrix using the Vihola algorithm and EMCEE sampler. The implementations were tested first by applying them to analytical benchmark functions and then on real datasets, showing a trend of improved performance resulting from the use of the EMCEE sampling strategy in conjunction with the application of Beta warping.File  Dimensione  Formato  

Manara_Noemi.pdf
accesso riservato
Dimensione
3.97 MB
Formato
Adobe PDF

3.97 MB  Adobe PDF 
The text of this website © Università degli studi di Padova. Full Text are published under a nonexclusive license. Metadata are under a CC0 License
https://hdl.handle.net/20.500.12608/45811