Chen Chaofeng ; Liu Wei ; Tan Xiao ; Wong Kwan-Yee K.

Abstract

Face sketch synthesis has made great progress in the past few years. Recentmethods based on deep neural networks are able to generate high qualitysketches from face photos. However, due to the lack of training data(photo-sketch pairs), none of such deep learning based methods can be appliedsuccessfully to face photos in the wild. In this paper, we propose asemi-supervised deep learning architecture which extends face sketch synthesisto handle face photos in the wild by exploiting additional face photos intraining. Instead of supervising the network with ground truth sketches, wefirst perform patch matching in feature space between the input photo andphotos in a small reference set of photo-sketch pairs. We then compose a pseudosketch feature representation using the corresponding sketch feature patches tosupervise our network. With the proposed approach, we can train our networksusing a small reference set of photo-sketch pairs together with a large facephoto dataset without ground truth sketches. Experiments show that our methodachieve state-of-the-art performance both on public benchmarks and face photosin the wild. Codes are available athttps://github.com/chaofengc/Face-Sketch-Wild.