Repository logo

Automatic Replication of Teleoperator Head Movements and Facial Expressions on a Humanoid Robot

Accepted version


Conference Object

Change log


Ondras, J 
Celiktutan, O 
Sariyanidi, E 


Robotic telepresence aims to create a physical presence for a remotely located human (teleoperator) by reproducing their verbal and nonverbal behaviours (e.g. speech, gestures, facial expressions) on a robotic platform. In this work, we propose a novel teleoperation system that combines the replication of facial expressions of emotions (neutral, disgust, happiness, and surprise) and head movements on the fly on the humanoid robot Nao. Robots' expression of emotions is constrained by their physical and behavioural capabilities. As the Nao robot has a static face, we use the LEDs located around its eyes to reproduce the teleoperator expressions of emotions. Using a web camera, we computationally detect the facial action units and measure the head pose of the operator. The emotion to be replicated is inferred from the detected action units by a neural network. Simultaneously, the measured head motion is smoothed and bounded to the robot's physical limits by applying a constrained-state Kalman filter. In order to evaluate the proposed system, we conducted a user study by asking 28 participants to use the replication system by displaying facial expressions and head movements while being recorded by a web camera. Subsequently, 18 external observers viewed the recorded clips via an online survey and assessed the quality of the robot's replication of the participants' behaviours. Our results show that the proposed teleoperation system can successfully communicate emotions and head movements, resulting in a high agreement among the external observers (ICC_E = 0.91, ICC_HP = 0.72).



HRI, telepresence, facial expressions, head movement, emotions, robotic replication

Journal Title

RO-MAN 2017 - 26th IEEE International Symposium on Robot and Human Interactive Communication

Conference Name

2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN)

Journal ISSN


Volume Title


Engineering and Physical Sciences Research Council (EP/L00416X/1)
This work was funded by the EPSRC under its IDEAS Factory Sandpits call on Digital Personhood (Grant Ref· EP/L00416X/1).