Zhu Xiangyu ; Liu Xiaoming ; Lei Zhen ; Li Stan Z.

Abstract

Face alignment, which fits a face model to an image and extracts the semanticmeanings of facial pixels, has been an important topic in the computer visioncommunity. However, most algorithms are designed for faces in small to mediumposes (yaw angle is smaller than 45 degrees), which lack the ability to alignfaces in large poses up to 90 degrees. The challenges are three-fold. Firstly,the commonly used landmark face model assumes that all the landmarks arevisible and is therefore not suitable for large poses. Secondly, the faceappearance varies more drastically across large poses, from the frontal view tothe profile view. Thirdly, labelling landmarks in large poses is extremelychallenging since the invisible landmarks have to be guessed. In this paper, wepropose to tackle these three challenges in an new alignment framework termed3D Dense Face Alignment (3DDFA), in which a dense 3D Morphable Model (3DMM) isfitted to the image via Cascaded Convolutional Neural Networks. We also utilize3D information to synthesize face images in profile views to provide abundantsamples for training. Experiments on the challenging AFLW database show thatthe proposed approach achieves significant improvements over thestate-of-the-art methods.