Repository logo
 

Personality Recognition by Modelling Person-specific Cognitive Processes using Graph Representation.

Accepted version
Peer-reviewed

Type

Conference Object

Change log

Authors

Shao, Zilong 
Song, Siyang 
Jaiswal, Shashank 
Shen, Linlin 
Valstar, Michel F 

Abstract

Recent research shows that in dyadic and group interactions individuals’ nonverbal behaviours are influenced by the behaviours of their conversational partner(s). Therefore, in this work we hypothesise that during a dyadic interaction, the target subject's facial reactions are driven by two main factors: (i) their internal (person-specific) cognition, and (ii) the externalised nonverbal behaviours of their conversational partner. Subsequently, our novel proposition is to simulate and represent the target subject’s (i.e., the listener) cognitive process in the form of a person-specific CNN architecture whose input is the audio-visual non-verbal cues displayed by the conversational partner (i.e., the speaker), and the output is the target subject’s (i.e., the listener) facial reactions. We then undertake a search for the optimal CNN architecture whose results are used to create a person-specific graph representation for recognising the target subject’s personality. The graph representation, fortified with a novel end-to-end edge feature learning strategy, helps with retaining both the unique parameters of the person-specific CNN and the geometrical relationship between its layers. Consequently, the proposed approach is the first work that aims to recognize the true (self-reported) personality of a target subject (i.e., the listener) from the learned simulation of their cognitive process (i.e., parameters of the person-specific CNN). The experimental results show that the CNN architectures are well associated with target subjects' personality traits and the proposed approach clearly outperforms multiple existing approaches that predict personality directly from non-verbal behaviours. In light of these findings, this work opens up a new avenue of research for predicting and recognizing socio-emotional phenomena (personality, affect, engagement etc.) from simulations of person-specific cognitive processes.

Description

Keywords

Journal Title

ACM Multimedia

Conference Name

29th ACM International Conference on Multimedia

Journal ISSN

Volume Title

Publisher

ACM

Rights

All rights reserved
Sponsorship
Engineering and Physical Sciences Research Council (EP/R030782/1)
European Commission Horizon 2020 (H2020) Societal Challenges (826232)
European Union’s Horizon 2020 Research and Innovation Programme, under the WorkingAge Project (grant agreement No.826232)