Computational Recognition of Facial Expressions in Sculpture

Abstract

The art of understating emotions across different cultures is a subjective experience; however, what cross-cuts the cultural expression is our human neuro-physiological ability to communicate our feelings via facial expressions. Combined with AI know-how, such capacity provides us with unique opportunities to trace the artists’ intentions to communicate particular emotions to the viewer, from those in the deep past c. 25,000 years ago to the contemporary sculpture. Here, we present a computational approach to analyzing facial expressions depicted in artwork of numerous regions, specifically sculptures. We collected a large dataset of sculptures’ faces from online collections of various museums and used the existing methods to assign labels to each image. Each instance may have more than one label predicted by different methods, so we treated facial expression recognition as a multi-label classification problem. We designed deep learning-based frameworks using different backbones to categorize facial expressions in sculptures. We also implemented GradCAM to visualize the attributes in each image contributing the most to the predicted labels.