Scalable Gaussian process inference using variational methods
View / Open Files
Advisors
Ghahramani, Zoubin
Date
2017-03-25Awarding Institution
University of Cambridge
Author Affiliation
Engineering
Qualification
Doctor of Philosophy (PhD)
Language
English
Type
Thesis
Metadata
Show full item recordCitation
Matthews, A. G. d. G. (2017). Scalable Gaussian process inference using variational methods (Doctoral thesis). https://doi.org/10.17863/CAM.25348
Abstract
Gaussian processes can be used as priors on functions. The need for a flexible, principled,
probabilistic model of functional relations is common in practice. Consequently, such an
approach is demonstrably useful in a large variety of applications.
Two challenges of Gaussian process modelling are often encountered. These are dealing
with the adverse scaling with the number of data points and the lack of closed form posteriors
when the likelihood is non-Gaussian. In this thesis, we study variational inference as a
framework for meeting these challenges.
An introductory chapter motivates the use of stochastic processes as priors, with a
particular focus on Gaussian process modelling. A section on variational inference reviews
the general definition of Kullback-Leibler divergence. The concept of prior conditional
matching that is used throughout the thesis is contrasted to classical approaches to obtaining
tractable variational approximating families.
Various theoretical issues arising from the application of variational inference to the
infinite dimensional Gaussian process setting are settled decisively. From this theory we
are able to give a new argument for existing approaches to variational regression that settles
debate about their applicability. This view on these methods justifies the principled extensions
found in the rest of the work.
The case of scalable Gaussian process classification is studied, both for its own merits
and as a case study for non-Gaussian likelihoods in general. Using the resulting algorithms
we find credible results on datasets of a scale and complexity that was not possible before
our work. An extension to include Bayesian priors on model hyperparameters is studied
alongside a new inference method that combines the benefits of variational sparsity and
MCMC methods. The utility of such an approach is shown on a variety of example modelling
tasks.
We describe GPflow, a new Gaussian process software library that uses TensorFlow.
Implementations of the variational algorithms discussed in the rest of the thesis are included
as part of the software. We discuss the benefits of GPflow when compared to other similar
software. Increased computational speed is demonstrated in relevant, timed, experimental
comparisons.
Keywords
Gaussian process, Variational inference, Machine learning, Statistics, Bayesian inference
Sponsorship
EPSRC grant EP/I036575/1
Identifiers
This record's DOI: https://doi.org/10.17863/CAM.25348
Rights
All rights reserved, All Rights Reserved, Included TensorFlow source file has the Apache 2.0 license. Copyright is held by Google Inc. The license header reads:
/* Copyright 2015 Google Inc. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
TensorFlow is a trademark of Google Inc.
Licence URL: https://www.rioxx.net/licenses/all-rights-reserved/
Statistics
Total file downloads (since January 2020). For more information on metrics see the
IRUS guide.