Facial attributes are important since they provide a detailed description and determine the visual appearance of human faces. In this paper, we aim at converting a face image to a sketch while simultaneously generating facial attributes. To this end, we propose a novel Attribute-Guided Sketch Generative Adversarial Network (ASGAN) which is an end-to-end framework and contains two pairs of generators and discriminators, one of which is used to generate faces with attributes while the other one is employed for image-to-sketch translation. The two generators form a W-shaped network (W-net) and they are trained jointly with a weight-sharing constraint. Additionally, we also propose two novel discriminators, the residual one focusing on attribute generation and the triplex one helping to generate realistic looking sketches. To validate our model, we have created a new large dataset with 8,804 images, named the Attribute Face Photo & Sketch (AFPS) dataset which is the first dataset containing attributes associated to face sketch images. The experimental results demonstrate that the proposed network (i) generates more photo-realistic faces with sharper facial attributes than baselines and (ii) has good generalization capability on different generative tasks.
Based on CGANs, @cite_46 have developed a generic framework Pix2pix'', which is suitable for different generative tasks. In Pix2pix, one conditional image is adopted as a reference during the training time. The generator in Pix2pix is a U-net, which tries to synthesize a fake image conditioned on the given conditional image in order to fool the discriminator, while the discriminator tries to identify the fake image by comparing it with the corresponding target image. Under these settings, the discriminator takes the pairs of images as input. The U-net is actually an Encoder-Decoder network with skip connection, in which the encoder consists of multiple convolution layers and the decoder consists of multiple deconvolution layers. @cite_46 added skip connections between each layer @math and layer @math which allows feature sharing between the encoder and decoder, where @math is the total number of layers. All channels at layer @math are simply concatenated with those at layer @math by the skip connections. @cite_46 shares a similar goal with us, but it cannot solve the face-to-attributed-sketch translation task since it cannot convert a face image to a sketch conditioned on external facial attributes while our ASGAN is specifically designed to tackle this task.
@cite_46
2963073614
We investigate conditional adversarial networks as a general-purpose solution to image-to-image translation problems. These networks not only learn the mapping from input image to output image, but also learn a loss function to train this mapping. This makes it possible to apply the same generic approach to problems that traditionally would require very different loss formulations. We demonstrate that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks. Moreover, since the release of the pix2pix software associated with this paper, hundreds of twitter users have posted their own artistic experiments using our system. As a community, we no longer hand-engineer our mapping functions, and this work suggests we can achieve reasonable results without handengineering our loss functions either.