Show simple item record

dc.contributor.authorBurt, Daviden
dc.contributor.authorRasmussen, Carlen
dc.contributor.authorvan der Wilk, Marken
dc.date.accessioned2019-10-25T23:30:23Z
dc.date.available2019-10-25T23:30:23Z
dc.identifier.issn2640-3498
dc.identifier.urihttps://www.repository.cam.ac.uk/handle/1810/298089
dc.description.abstractExcellent variational approximations to Gaussian process posteriors have been developed which avoid the O(N³) scaling with dataset size N. They reduce the computational cost to O(NM²), with M≪N being the number of inducing variables, which summarise the process. While the computational cost seems to be linear in N, the true complexity of the algorithm depends on how M must increase to ensure a certain quality of approximation. We address this by characterising the behavior of an upper bound on the KL divergence to the posterior. We show that with high probability the KL divergence can be made arbitrarily small by growing M more slowly than N. A particular case of interest is that for regression with normally distributed inputs in D-dimensions with the popular Squared Exponential kernel, M = O(log^DN) is sufficient. Our results show that as datasets grow, Gaussian process posteriors can truly be approximated cheaply, and provide a concrete rule for how to increase M in continual learning scenarios.
dc.publisherProceedings of Machine Learning Research
dc.rightsAttribution 4.0 International
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.titleRates of Convergence for Sparse Variational Gaussian Process Regressionen
dc.typeConference Object
prism.endingPage871
prism.publicationNameProceedings of the 36th International Conference on Machine Learningen
prism.startingPage862
prism.volume97en
dc.identifier.doi10.17863/CAM.45147
dcterms.dateAccepted2019-04-22en
rioxxterms.versionVoR
rioxxterms.licenseref.urihttp://www.rioxx.net/licenses/all-rights-reserveden
rioxxterms.licenseref.startdate2019-04-22en
dc.contributor.orcidRasmussen, Carl [0000-0001-8899-7850]
rioxxterms.typeConference Paper/Proceeding/Abstracten
cam.issuedOnline2019-06-26en
dc.identifier.urlhttp://proceedings.mlr.press/v97/en
pubs.conference-nameICML Thirty-sixth International Conference on Machine Learningen
pubs.conference-start-date2019-06-10en
cam.orpheus.successThu Nov 05 11:54:57 GMT 2020 - The item has an open VoR version.*
rioxxterms.freetoread.startdate2100-01-01


Files in this item

Thumbnail
Thumbnail
Thumbnail

This item appears in the following Collection(s)

Show simple item record

Attribution 4.0 International
Except where otherwise noted, this item's licence is described as Attribution 4.0 International