Advanced Bayesian Monte Carlo Methods for Inference and Control

Abstract

Monte Carlo methods are are an ubiquitous tool in modern statistics. Under the Bayesian paradigm, they are used for estimating otherwise intractable integrals arising when integrating a function $h$ with respect to a posterior distribution $\pi$. This thesis discusses several aspects of such Monte Carlo methods. The first discussion evolves around the problem of sampling from only almost everywhere differentiable distributions, a class of distributions which includes all log-concave posteriors. A new sampling method based on a second-order diffusion process is proposed, new theoretical results are proved, and extensive numerical illustrations elucidate the benefits and weaknesses of various methods applicable in these settings. In high-dimensional settings, one can exploit local structures of inverse problems to parallelise computations. This will be explored in both fully localisable problems, and problems where conditional independence of variables given some others holds only approximately. This thesis proposes two algorithms using parallelisation techniques, and shows their empirical performance on two localisable imaging problems. Another problem arises when defining function space priors over high-dimensional domains. The commonly used Karhunen-Loève priors suffer from bad dimensional scaling: they require an orthogonal basis of the function space, which can often be obtained as a product of one-dimensional basis functions. This leads to the number of parameters growing exponentially in the dimension $d$ of the function domain. The trace-class neural network prior, proposed in this thesis, scales more favourably in the dimension of the function's domain. This prior is a Bayesian neural network prior, where each weight and bias has an independent Gaussian prior, but with a key difference to existing Bayesian neural network priors: the variances decrease in the width of the network, such that the variances form a summable sequence and the infinite width limit neural network is well defined. As is shown in this thesis, the resulting posterior of the unknown function is amenable to sampling using Hilbert space Markov chain Monte Carlo methods. These sampling methods are favoured because they are stable under mesh-refinement, in the sense that the acceptance probability does not shrink to 0 as more parameters are introduced to better approximate the well-defined infinite limit. Both numerical illustrations and theoretical results show that these priors are competitive and have distinct advantages over other function space priors. These different function space priors are then used in stochastic control. To this end, a suitable likelihood for continuous value functions in a Bayesian approach to reinforcement learning is defined. This thesis proves that it can be used in conjunction with both the classical Karhunen-Loève prior and the proposed trace-class neural network prior. Numerical examples compare the resulting posteriors, and illustrate the new prior's performance and dimension robustness.