AULA-Caps: Lifecycle-Aware Capsule Networks for Spatio-Temporal Analysis of Facial Actions
IEEE International Conference on Automatic Face and Gesture Recognition (FG) 2021
MetadataShow full item record
Churamani, N., Kalkan, S., & Gunes, H. AULA-Caps: Lifecycle-Aware Capsule Networks for Spatio-Temporal Analysis of Facial Actions. IEEE International Conference on Automatic Face and Gesture Recognition (FG) 2021. https://doi.org/10.17863/CAM.78113
Most state-of-the-art approaches for Facial Action Unit (AU) detection rely on evaluating static frames, encoding a snapshot of heightened facial activity. In real-world interactions, however, facial expressions are more subtle and evolve over time requiring AU detection models to learn spatial as well as temporal information. In this work, we focus on both spatial and spatio-temporal features encoding the temporal evolution of facial AU activation. We propose the Action Unit Lifecycle- Aware Capsule Network (AULA-Caps) for AU detection using both frame and sequence-level features. While, at the frame- level, the capsule layers of AULA-Caps learn spatial feature primitives to determine AU activations, at the sequence-level, it learns temporal dependencies between contiguous frames by focusing on relevant spatio-temporal segments in the sequence. The learnt feature capsules are routed together such that the model learns to selectively focus on spatial or spatio-temporal information depending upon the AU lifecycle. The proposed model is evaluated on popular benchmarks, namely BP4D and GFT datasets, obtaining state-of-the-art results for both.
Affective Computing, Facial Action Units, Capsule Networks, Computer Vision, Machine Learning, Neural Networks
EPSRC grant EP/R513180/1 (ref. 2107412). EPSRC project ARoEQ under grant ref. EP/R030782/1. European Union’s Horizon 2020 research and innovation programme WorkingAge project under grant agreement No. 826232.
Engineering and Physical Sciences Research Council (EP/R030782/1)
European Commission Horizon 2020 (H2020) Societal Challenges (826232)
Embargo Lift Date
External DOI: https://doi.org/10.17863/CAM.78113
This record's URL: https://www.repository.cam.ac.uk/handle/1810/330668
All rights reserved
Licence URL: http://www.rioxx.net/licenses/all-rights-reserved