The difficulty of computing stable and accurate neural networks: On the barriers of deep learning and Smale's 18th problem.
View / Open Files
Publication Date
2022-03-22Journal Title
Proc Natl Acad Sci U S A
ISSN
0027-8424
Publisher
Proceedings of the National Academy of Sciences
Volume
119
Issue
12
Pages
e2107151119
Type
Article
This Version
VoR
Physical Medium
Print-Electronic
Metadata
Show full item recordCitation
Colbrook, M., Antun, V., & Hansen, A. (2022). The difficulty of computing stable and accurate neural networks: On the barriers of deep learning and Smale's 18th problem.. Proc Natl Acad Sci U S A, 119 (12), e2107151119. https://doi.org/10.1073/pnas.2107151119
Abstract
SignificanceInstability is the Achilles' heel of modern artificial intelligence (AI) and a paradox, with training algorithms finding unstable neural networks (NNs) despite the existence of stable ones. This foundational issue relates to Smale's 18th mathematical problem for the 21st century on the limits of AI. By expanding methodologies initiated by Gödel and Turing, we demonstrate limitations on the existence of (even randomized) algorithms for computing NNs. Despite numerous existence results of NNs with great approximation properties, only in specific cases do there also exist algorithms that can compute them. We initiate a classification theory on which NNs can be trained and introduce NNs that-under suitable conditions-are robust to perturbations and exponentially accurate in the number of hidden layers.
Keywords
AI and deep learning, Smale’s 18th problem, inverse problems, solvability complexity index hierarchy, stability and accuracy
Identifiers
External DOI: https://doi.org/10.1073/pnas.2107151119
This record's URL: https://www.repository.cam.ac.uk/handle/1810/335732
Rights
Attribution-NonCommercial-NoDerivatives 4.0 International
Licence URL: https://creativecommons.org/licenses/by-nc-nd/4.0/
Statistics
Total file downloads (since January 2020). For more information on metrics see the
IRUS guide.