Deep convolutional networks as shallow Gaussian processes
View / Open Files
Authors
Garriga-Alonso, A
Aitchison, L
Rasmussen, CE
Publication Date
2018-08-16Journal Title
7th International Conference on Learning Representations, ICLR 2019
Type
Article
Metadata
Show full item recordCitation
Garriga-Alonso, A., Aitchison, L., & Rasmussen, C. (2018). Deep convolutional networks as shallow Gaussian processes. 7th International Conference on Learning Representations, ICLR 2019 https://doi.org/10.17863/CAM.42340
Abstract
We show that the output of a (residual) convolutional neural network (CNN)
with an appropriate prior over the weights and biases is a Gaussian process
(GP) in the limit of infinitely many convolutional filters, extending similar
results for dense networks. For a CNN, the equivalent kernel can be computed
exactly and, unlike "deep kernels", has very few parameters: only the
hyperparameters of the original CNN. Further, we show that this kernel has two
properties that allow it to be computed efficiently; the cost of evaluating the
kernel for a pair of images is similar to a single forward pass through the
original CNN with only one filter per layer. The kernel equivalent to a
32-layer ResNet obtains 0.84% classification error on MNIST, a new record for
GPs with a comparable number of parameters.
Identifiers
This record's DOI: https://doi.org/10.17863/CAM.42340
This record's URL: https://www.repository.cam.ac.uk/handle/1810/295286
Rights
All rights reserved
Licence:
http://www.rioxx.net/licenses/all-rights-reserved
Statistics
Total file downloads (since January 2020). For more information on metrics see the
IRUS guide.
Recommended or similar items
The current recommendation prototype on the Apollo Repository will be turned off on 03 February 2023. Although the pilot has been fruitful for both parties, the service provider IKVA is focusing on horizon scanning products and so the recommender service can no longer be supported. We recognise the importance of recommender services in supporting research discovery and are evaluating offerings from other service providers. If you would like to offer feedback on this decision please contact us on: support@repository.cam.ac.uk