The difficulty of computing stable and accurate neural networks: On the barriers of deep learning and Smale's 18th problem.
Proc Natl Acad Sci U S A
Proceedings of the National Academy of Sciences
MetadataShow full item record
Colbrook, M., Antun, V., & Hansen, A. (2022). The difficulty of computing stable and accurate neural networks: On the barriers of deep learning and Smale's 18th problem.. Proc Natl Acad Sci U S A, 119 (12), e2107151119. https://doi.org/10.1073/pnas.2107151119
SignificanceInstability is the Achilles' heel of modern artificial intelligence (AI) and a paradox, with training algorithms finding unstable neural networks (NNs) despite the existence of stable ones. This foundational issue relates to Smale's 18th mathematical problem for the 21st century on the limits of AI. By expanding methodologies initiated by Gödel and Turing, we demonstrate limitations on the existence of (even randomized) algorithms for computing NNs. Despite numerous existence results of NNs with great approximation properties, only in specific cases do there also exist algorithms that can compute them. We initiate a classification theory on which NNs can be trained and introduce NNs that-under suitable conditions-are robust to perturbations and exponentially accurate in the number of hidden layers.
AI and deep learning, Smale’s 18th problem, inverse problems, solvability complexity index hierarchy, stability and accuracy
External DOI: https://doi.org/10.1073/pnas.2107151119
This record's URL: https://www.repository.cam.ac.uk/handle/1810/335732
Attribution-NonCommercial-NoDerivatives 4.0 International
Licence URL: https://creativecommons.org/licenses/by-nc-nd/4.0/