Yang Tao ; Ren Peiran ; Xie Xuansong ; Zhang Lei

Abstract

Blind face restoration (BFR) from severely degraded face images in the wildis a very challenging problem. Due to the high illness of the problem and thecomplex unknown degradation, directly training a deep neural network (DNN)usually cannot lead to acceptable results. Existing generative adversarialnetwork (GAN) based methods can produce better results but tend to generateover-smoothed restorations. In this work, we propose a new method by firstlearning a GAN for high-quality face image generation and embedding it into aU-shaped DNN as a prior decoder, then fine-tuning the GAN prior embedded DNNwith a set of synthesized low-quality face images. The GAN blocks are designedto ensure that the latent code and noise input to the GAN can be respectivelygenerated from the deep and shallow features of the DNN, controlling the globalface structure, local face details and background of the reconstructed image.The proposed GAN prior embedded network (GPEN) is easy-to-implement, and it cangenerate visually photo-realistic results. Our experiments demonstrated thatthe proposed GPEN achieves significantly superior results to state-of-the-artBFR methods both quantitatively and qualitatively, especially for therestoration of severely degraded face images in the wild. The source code andmodels can be found at https://github.com/yangxy/GPEN.