An Empirical Bayes approach to ARX estimation

Empirical Bayes Inference is based on estimates of the a priori parameter distribution obtained from the observed data. There are claims in the literature that in certain cases this technique may even be superior to Maximum Likelihood. In our opinion however this claim does not seem to be substantiated in sufficient generality as it may depend on various factors such as model class, data length, randomly varying parameters etc. In this paper we compare Empirical Bayes estimators with a standard estimation technique for linear Autoregressive models with inputs (called ARX models) for time series. Such a comparison, can only make sense for a (realistic) finite data length; In this setting, it turns out that Empirical Bayes tends indeed to behave slightly better and so also in the case of slowly varying random parameters.