Show simple item record

dc.contributor.authorHavasi, Marton
dc.contributor.authorSnoek, Jasper
dc.contributor.authorTran, Dustin
dc.contributor.authorGordon, Jonathan
dc.contributor.authorHernández-Lobato, José Miguel
dc.date.accessioned2021-11-12T18:55:39Z
dc.date.available2021-11-12T18:55:39Z
dc.date.issued2021-11-08
dc.identifier.issn1099-4300
dc.identifier.urihttps://www.repository.cam.ac.uk/handle/1810/330598
dc.description.abstractVariational inference is an optimization-based method for approximating the posterior distribution of the parameters in Bayesian probabilistic models. A key challenge of variational inference is to approximate the posterior with a distribution that is computationally tractable yet sufficiently expressive. We propose a novel method for generating samples from a highly flexible variational approximation. The method starts with a coarse initial approximation and generates samples by refining it in selected, local regions. This allows the samples to capture dependencies and multi-modality in the posterior, even when these are absent from the initial approximation. We demonstrate theoretically that our method always improves the quality of the approximation (as measured by the evidence lower bound). In experiments, our method consistently outperforms recent variational inference methods in terms of log-likelihood and ELBO across three example tasks: the Eight-Schools example (an inference task in a hierarchical model), training a ResNet-20 (Bayesian inference in a large neural network), and the Mushroom task (posterior sampling in a contextual bandit problem).
dc.languageen
dc.publisherMDPI AG
dc.subjectbayesian inference
dc.subjectvariational inference
dc.subjectdeep neural networks
dc.subjectcontextual bandits
dc.titleSampling the Variational Posterior with Local Refinement.
dc.typeArticle
dc.date.updated2021-11-12T18:55:38Z
prism.issueIdentifier11
prism.publicationNameEntropy (Basel)
prism.volume23
dc.identifier.doi10.17863/CAM.78042
dcterms.dateAccepted2021-11-03
rioxxterms.versionofrecord10.3390/e23111475
rioxxterms.versionVoR
rioxxterms.licenseref.urihttps://creativecommons.org/licenses/by/4.0/
dc.identifier.eissn1099-4300
cam.issuedOnline2021-11-08


Files in this item

Thumbnail
Thumbnail
Thumbnail

This item appears in the following Collection(s)

Show simple item record