Computational Audio Modelling for Robot-Assisted Assessment of Children’s Mental Wellbeing

Abbasi, NI 
Spitale, M 
Anderson, J 
Ford, T 
Jones, PB 

Thumbnail Image
Conference Object
Change log

Robots endowed with the capability of assessing the mental wellbeing of children have a great potential to promote their mental health. However, very few works have explored the computational modelling of children’s mental wellbeing, which remains an open research challenge. This paper presents the first attempt to computationally assess children’s wellbeing during child-robot interactions via audio analysis. We collected a novel dataset of 26 children (8-13 y.o.) who interacted with a Nao robot to perform a verbal picture-based task. Data was collected by audio-video recording of the experiment session. The Short Mood and Feelings Questionnaire (SMFQ) was used to label the participants into two groups: (1)“higher wellbeing” (child SMFQ score <= SMFQ median), and (2) “lower wellbeing” (child SMFQ score > SMFQ median). We extracted audio features from these HRI interactions and trained and compared the performances of eight classical machine learning techniques across three cross-validation approaches: (1) 10 repetitions of 5-fold, (2) leave-one-child-out, and (3) leave-one-picture-out. We have also computed and analysed the sentiment of the audio transcriptions using the ROBERTa model. Our experimental results show that: (i) speech features are reliable for assessing children’s mental wellbeing, but they may not be sufficient on their own, and (ii) verbal information, specifically the sentiment that a picture elicited in children, may impact the children’s responses.

Publication Date
Online Publication Date
Acceptance Date
46 Information and Computing Sciences, 4608 Human-Centred Computing, Mental Health, Brain Disorders, Behavioral and Social Science, Pediatric, Mental health, 3 Good Health and Well Being
Journal Title
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Journal ISSN
Volume Title
Springer Nature Switzerland
Engineering and Physical Sciences Research Council (EP/R030782/1)
National Institute for Health Research (NIHR200177)
National Institute for Health Research (IS-BRC-1215-20014)
This work was supported by the University of Cambridge’s OHMC Small Equipment Funding. N. I. Abbasi is supported by the W.D. Armstrong Trust PhD Studentship and the Cambridge Trusts. M. Spitale and H. Gunes are supported by the EPSRC project ARoEQ under grant ref. EP/R030782/1. All research at the Department of Psychiatry in the University of Cambridge is supported by the NIHR Cambridge Biomedical Research Centre (BRC-1215-20014, particularly T. Ford) and NIHR Applied Research Collaboration East of England (P. Jones, J. Anderson).