Advances in Practical Markov Chain Monte Carlo and Probabilistic Programming

Abstract

Estimating expectations under probability distributions is fundamental in scientific computing, with applications in statistical physics, Bayesian inference, and beyond. Markov Chain Monte Carlo (MCMC) methods enable such computations but suffer from challenges like autocorrelation, slow convergence, and manual parameter tuning. Similarly, Bayesian inference workflows involve iterative model specification and validation, which can be complex and time-consuming. This thesis makes progress on these challenges in two parts. First, we develop methods to remove burn-in bias in MCMC, improving efficiency and enabling parallel computation, with a focus on Hamiltonian Monte Carlo. Second, we advance probabilistic programming by contributing to the design and implementation of Turing.jl, a Julia-based language that simplifies Bayesian modelling and inference. Together, these contributions enhance the robustness, scalability, and accessibility of Bayesian computation.