Learning Deep Mixtures of Gaussian Process Experts Using Sum-Product Networks
View / Open Files
Conference Name
35th International Conference on Machine Learning
Volume
PMLR 80, 2018
Language
English
Type
Conference Object
This Version
AM
Metadata
Show full item recordCitation
Trapp, M., Peharz, R., Rasmussen, C., & Pernkopf, F. (2018). Learning Deep Mixtures of Gaussian Process Experts Using Sum-Product Networks. 35th International Conference on Machine Learning. PMLR 80, 2018 https://doi.org/10.17863/CAM.42182
Abstract
While Gaussian processes (GPs) are the method of choice for regression tasks,
they also come with practical difficulties, as inference cost scales cubic in
time and quadratic in memory. In this paper, we introduce a natural and
expressive way to tackle these problems, by incorporating GPs in sum-product
networks (SPNs), a recently proposed tractable probabilistic model allowing
exact and efficient inference. In particular, by using GPs as leaves of an SPN
we obtain a novel flexible prior over functions, which implicitly represents an
exponentially large mixture of local GPs. Exact and efficient posterior
inference in this model can be done in a natural interplay of the inference
mechanisms in GPs and SPNs. Thereby, each GP is -- similarly as in a mixture of
experts approach -- responsible only for a subset of data points, which
effectively reduces inference cost in a divide and conquer fashion. We show
that integrating GPs into the SPN framework leads to a promising probabilistic
regression model which is: (1) computational and memory efficient, (2) allows
efficient and exact posterior inference, (3) is flexible enough to mix
different kernel functions, and (4) naturally accounts for non-stationarities
in time series. In a variate of experiments, we show that the SPN-GP model can
learn input dependent parameters and hyper-parameters and is on par with or
outperforms the traditional GPs as well as state of the art approximations on
real-world data.
Keywords
cs.LG, cs.LG, stat.ML
Sponsorship
European Commission Horizon 2020 (H2020) Marie Sklodowska-Curie actions (797223)
Identifiers
External DOI: https://doi.org/10.17863/CAM.42182
This record's URL: https://www.repository.cam.ac.uk/handle/1810/295111
Rights
Licence:
http://www.rioxx.net/licenses/all-rights-reserved
Statistics
Total file downloads (since January 2020). For more information on metrics see the
IRUS guide.