Repository logo
 

Learning Deep Mixtures of Gaussian Process Experts Using Sum-Product Networks

Accepted version
Peer-reviewed

Type

Conference Object

Change log

Authors

Trapp, Martin 
Rasmussen, Carl E 
Pernkopf, Franz 

Abstract

While Gaussian processes (GPs) are the method of choice for regression tasks, they also come with practical difficulties, as inference cost scales cubic in time and quadratic in memory. In this paper, we introduce a natural and expressive way to tackle these problems, by incorporating GPs in sum-product networks (SPNs), a recently proposed tractable probabilistic model allowing exact and efficient inference. In particular, by using GPs as leaves of an SPN we obtain a novel flexible prior over functions, which implicitly represents an exponentially large mixture of local GPs. Exact and efficient posterior inference in this model can be done in a natural interplay of the inference mechanisms in GPs and SPNs. Thereby, each GP is -- similarly as in a mixture of experts approach -- responsible only for a subset of data points, which effectively reduces inference cost in a divide and conquer fashion. We show that integrating GPs into the SPN framework leads to a promising probabilistic regression model which is: (1) computational and memory efficient, (2) allows efficient and exact posterior inference, (3) is flexible enough to mix different kernel functions, and (4) naturally accounts for non-stationarities in time series. In a variate of experiments, we show that the SPN-GP model can learn input dependent parameters and hyper-parameters and is on par with or outperforms the traditional GPs as well as state of the art approximations on real-world data.

Description

Keywords

cs.LG, cs.LG, stat.ML

Journal Title

Conference Name

35th International Conference on Machine Learning

Journal ISSN

Volume Title

PMLR 80, 2018

Publisher

Sponsorship
European Commission Horizon 2020 (H2020) Marie Sklodowska-Curie actions (797223)