Show simple item record

dc.contributor.authorWu, J
dc.contributor.authorDang, Ting
dc.contributor.authorSethu, V
dc.contributor.authorAmbikairajah, E
dc.date.accessioned2022-01-10T12:48:44Z
dc.date.available2022-01-10T12:48:44Z
dc.date.issued2021-12-23
dc.date.submitted2021-08-31
dc.identifier.issn2624-9898
dc.identifier.urihttps://www.repository.cam.ac.uk/handle/1810/332530
dc.description.abstract<jats:p>People perceive emotions via multiple cues, predominantly speech and visual cues, and a number of emotion recognition systems utilize both audio and visual cues. Moreover, the perception of static aspects of emotion (speaker's arousal level is high/low) and the dynamic aspects of emotion (speaker is becoming more aroused) might be perceived via different expressive cues and these two aspects are integrated to provide a unified sense of emotion state. However, existing multimodal systems only focus on single aspect of emotion perception and the contributions of different modalities toward modeling static and dynamic emotion aspects are not well explored. In this paper, we investigate the relative salience of audio and video modalities to emotion state prediction and emotion change prediction using a Multimodal Markovian affect model. Experiments conducted in the RECOLA database showed that audio modality is better at modeling the emotion state of arousal and video for emotion state of valence, whereas audio shows superior advantages over video in modeling emotion changes for both arousal and valence.</jats:p>
dc.languageen
dc.publisherFrontiers Media SA
dc.subjectComputer Science
dc.subjectemotion recognition
dc.subjectmultimodal
dc.subjectemotion dynamics
dc.subjectordinal data
dc.subjectmachine learning
dc.titleMultimodal Affect Models: An Investigation of Relative Salience of Audio and Visual Cues for Emotion Prediction
dc.typeArticle
dc.date.updated2022-01-10T12:48:43Z
prism.publicationNameFrontiers in Computer Science
prism.volume3
dc.identifier.doi10.17863/CAM.79980
dcterms.dateAccepted2021-12-02
rioxxterms.versionofrecord10.3389/fcomp.2021.767767
rioxxterms.versionVoR
rioxxterms.licenseref.urihttp://creativecommons.org/licenses/by/4.0/
dc.contributor.orcidDang, Ting [0000-0003-3806-1493]
dc.identifier.eissn2624-9898
cam.issuedOnline2021-12-23


Files in this item

Thumbnail
Thumbnail
Thumbnail
Thumbnail

This item appears in the following Collection(s)

Show simple item record