Characterising the area under the curve loss function landscape
Publication Date
2022Journal Title
Machine Learning: Science and Technology
ISSN
2632-2153
Publisher
IOP Publishing
Volume
3
Issue
1
Language
en
Type
Article
This Version
VoR
Metadata
Show full item recordCitation
Niroomand, M., Cafolla, C., Morgan, J., & Wales, D. (2022). Characterising the area under the curve loss function landscape. Machine Learning: Science and Technology, 3 (1) https://doi.org/10.1088/2632-2153/ac49a9
Abstract
<jats:title>Abstract</jats:title>
<jats:p>One of the most common metrics to evaluate neural network classifiers is the area under the receiver operating characteristic curve (AUC). However, optimisation of the AUC as the loss function during network training is not a standard procedure. Here we compare minimising the cross-entropy (CE) loss and optimising the AUC directly. In particular, we analyse the loss function landscape (LFL) of approximate AUC (appAUC) loss functions to discover the organisation of this solution space. We discuss various surrogates for AUC approximation and show their differences. We find that the characteristics of the appAUC landscape are significantly different from the CE landscape. The approximate AUC loss function improves testing AUC, and the appAUC landscape has substantially more minima, but these minima are less robust, with larger average Hessian eigenvalues. We provide a theoretical foundation to explain these results. To generalise our results, we lastly provide an overview of how the LFL can help to guide loss function analysis and selection.</jats:p>
Keywords
Paper, area under the curve, loss function landscape, basin hopping, alternative loss function, loss function
Sponsorship
EPSRC
Downing College, Cambridge
Interdisciplinary Institute for Artificial Intelligence at 3iA Cote d'Azur
Funder references
Engineering and Physical Sciences Research Council (EP/N035003/1)
Identifiers
mlstac49a9, ac49a9, mlst-100459.r1
External DOI: https://doi.org/10.1088/2632-2153/ac49a9
This record's URL: https://www.repository.cam.ac.uk/handle/1810/333218
Rights
Licence:
http://creativecommons.org/licenses/by/4.0
Statistics
Total file downloads (since January 2020). For more information on metrics see the
IRUS guide.
Recommended or similar items
The current recommendation prototype on the Apollo Repository will be turned off on 03 February 2023. Although the pilot has been fruitful for both parties, the service provider IKVA is focusing on horizon scanning products and so the recommender service can no longer be supported. We recognise the importance of recommender services in supporting research discovery and are evaluating offerings from other service providers. If you would like to offer feedback on this decision please contact us on: support@repository.cam.ac.uk