Repository logo
 

Predicting personal traits from facial images using convolutional neural networks augmented with facial landmark information

Accepted version
Peer-reviewed

Type

Article

Change log

Authors

Lewenberg, Y 
Bachrach, Y 
Shankar, S 
Criminisi, A 

Abstract

We consider the task of predicting various traits of a person given an image of their face. We estimate both objective traits, such as gender, ethnicity and hair-color; as well as subjective traits, such as the emotion a person expresses or whether he is humorous or attractive. For sizeable experimentation, we contribute a new Face Attributes Dataset (FAD), having roughly 200,000 attribute labels for the above traits, for over 10,000 facial images. Due to the recent surge of research on Deep Convolutional Neural Networks (CNNs), we begin by using a CNN architecture for estimating facial attributes and show that they indeed provide an impressive baseline performance. To further improve performance, we propose a novel approach that incorporates facial landmark information for input images as an additional channel, helping the CNN learn better attribute-specific features so that the landmarks across various training images hold correspondence. We empirically analyse the performance of our method, showing consistent improvement over the baseline across traits.

Description

Keywords

Journal Title

Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence

Conference Name

Journal ISSN

Volume Title

Publisher

AAAI Press

Publisher DOI

Sponsorship
Microsoft Research