Computational methods for Bayesian model assessment

This thesis studies computational tools for Bayesian modelling workflow. The focus is on two important areas. The first part of the thesis discusses the use of importance sampling for model assessment. Importance sampling is a generic computational tool that is used in many different applications. In model assessment, it is commonly used to speed up the computation of leave-one-out cross-validation. This thesis studies techniques for adapting the proposal distribution of importance sampling in order to improve the accuracy of leave-one-out cross-validation computations. To accomplish this, this thesis introduces a generic method for adapting a proposal distribution implicitly, which is thus applicable to a variety of complex distributions. The thesis also studies the special characteristics of adaptive importance sampling for self-normalized importance sampling, which is commonly used, for example, with probabilistic programming tools. The thesis also studies importance sampling techniques for analyzing the sensitivity of Bayesian model posteriors to the choice of prior and likelihood. These methods can be useful for detecting various modelling issues, such as overly influential prior distributions. Importance sampling is beneficial for this purpose due to its simplicity, and these methods can be automated and used as part of Bayesian modelling workflow easily. The second part of this thesis studies the assessment of the importance of variables in supervised learning. A common approach for generic variable importance assessment is to analyze the predictions of the model in real or transformed observations. This thesis presents methods for incorporating the predictive uncertainty of the model in such evaluation of variable importance. This thesis introduces the concept of uncertainty-aware sensitivity that generalizes sensitivity analysis from single predictions to probability distributions. The thesis develops an analytical framework for uncertainty-aware sensitivity as well as practical algorithms for its computation with different supervised learning models. The method is utilized to assess the importance of variables and variable interactions in Gaussian process models applied to different settings. The uncertainty-aware methods are especially useful for Gaussian processes and other flexible models where both variable importance and predictive uncertainty can vary significantly depending on the point of evaluation. The contributions of this thesis are mostly methodological, and an integral part of the contribution is code written as part of the publications. While the thesis does not focus on any single application, the applicability of the studied methods is demonstrated with a variety of freely available data sets from different fields.