Repository logo
 

Advances in approximate inference: combining VI and MCMC and improving on Stein discrepancy

Simple item page
cam.depositDate2022-06-21
cam.restrictionthesis_access_open
cam.supervisorHernandez Lobato, Jose
dc.contributor.authorGong, Wenbo
dc.date.accessioned2022-06-22T16:40:07Z
dc.date.available2022-06-22T16:40:07Z
dc.date.submitted2021-09-01
dc.date.updated2022-06-21T08:59:25Z
dc.description.abstractIn the modern world, machine learning, including deep learning, has become an indispensable part of many intelligent systems, helping people automate the decision-making process. For certain applications (e.g. health care services), reliable predictions and trustworthy decisions are crucial factors to be considered when deploying a machine learning system. In other words, machine learning models should be able to reason under uncertainty. Bayesian inference, powered by the probabilistic framework, is believed to be a principled way to incorporate uncertainty into the decision making process. The difficulty of applying Bayesian inference in practice is rooted in the intractability of computing posterior probabilities. Approximate inference provides an alternative workaround by providing a tractable estimate of posterior probabilities. The performance of Bayesian inference, especially in Bayesian deep learning, crucially depends on the quality of the chosen approximate inference algorithm in terms of its accuracy and scalability. Particularly, variational inference (VI) and Markov Chain Monte Carlo (MCMC) are two major techniques with their own merits and limitations. In the first part of the thesis, we aim to design efficient approximate inference algorithms by combining VI and MCMC (particularly, stochastic gradient MCMC (SG-MCMC)). The first proposed algorithm is called partial amortised inference, which leverages SG-MCMC to improve VI’s accuracy. The uncertainty quantification provided by this inference allows us to solve a practical problem: How to train a VAE-like generative model with insufficient training data. The second algorithm, named Meta-SGMCMC, aims at improving the efficiency of SG-MCMC by automating its dynamics design through meta learning and VI. In the second part of the thesis, we shift our focus to a promising future: Stein discrepancy, which greatly expands the choice of approximating distributions compared to the Kullback-Leibler (KL) divergence. We aim to improve on it by addressing the well-known curse-of-dimensionality problem of its scalable variant: kernelized Stein discrepancy. Inspired by the ’slicing’ idea, we propose a new discrepancy family called sliced kernelized Stein discrepancy that is robust to increasing dimensions, along with two theoretically verified downstream applications.
dc.description.sponsorshipCambridge Trust and Chinese Scholarship Council
dc.identifier.doi10.17863/CAM.85705
dc.identifier.urihttps://www.repository.cam.ac.uk/handle/1810/338296
dc.language.isoeng
dc.publisher.institutionUniversity of Cambridge
dc.rightsAll Rights Reserved
dc.rights.urihttps://www.rioxx.net/licenses/all-rights-reserved/
dc.subjectMachine Learning
dc.subjectComputer Science
dc.subjectStatistics
dc.subjectApproximate Inference
dc.titleAdvances in approximate inference: combining VI and MCMC and improving on Stein discrepancy
dc.typeThesis
dc.type.qualificationlevelDoctoral
dc.type.qualificationnameDoctor of Philosophy (PhD)
pubs.licence-display-nameApollo Repository Deposit Licence Agreement
pubs.licence-identifierapollo-deposit-licence-2-1
rioxxterms.licenseref.urihttps://www.rioxx.net/licenses/all-rights-reserved/
rioxxterms.typeThesis

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
PhD_Thesis_Print.pdf
Size:
7.53 MB
Format:
Adobe Portable Document Format
Description:
Thesis
Licence
https://www.rioxx.net/licenses/all-rights-reserved/
Download

Collections

Theses - Engineering