Wang Lidan Sindagi Vishwanath A. Patel Vishal M.

Abstract

Synthesizing face sketches from real photos and its inverse have manyapplications. However, photo/sketch synthesis remains a challenging problem dueto the fact that photo and sketch have different characteristics. In this work,we consider this task as an image-to-image translation problem and explore therecently popular generative models (GANs) to generate high-quality realisticphotos from sketches and sketches from photos. Recent GAN-based methods haveshown promising results on image-to-image translation problems andphoto-to-sketch synthesis in particular, however, they are known to havelimited abilities in generating high-resolution realistic images. To this end,we propose a novel synthesis framework called Photo-Sketch Synthesis usingMulti-Adversarial Networks, (PS2-MAN) that iteratively generates low resolutionto high resolution images in an adversarial way. The hidden layers of thegenerator are supervised to first generate lower resolution images followed byimplicit refinement in the network to generate higher resolution images.Furthermore, since photo-sketch synthesis is a coupled/paired translationproblem, we leverage the pair information using CycleGAN framework. Both ImageQuality Assessment (IQA) and Photo-Sketch Matching experiments are conducted todemonstrate the superior performance of our framework in comparison to existingstate-of-the-art solutions. Code available at:https://github.com/lidan1/PhotoSketchMAN.