Portrait editing is a popular subject in photo manipulation. The Generative Adversarial Network (GAN) advances the generating of realistic faces and allows more face editing. In this paper, we argue about three issues in existing techniques: diversity, quality, and controllability for portrait synthesis and editing. To address these issues, we propose a novel end-to-end learning framework that leverages conditional GANs guided by provided face masks for generating faces. The framework learns feature embeddings for every face component (e.g., mouth, hair, eye), separately, contributing to better correspondences for image translation, and local face editing. With the mask, our network is available to many applications, like face synthesis driven by mask, face Swap+ (including hair in swapping), and local manipulation. It can also boost the performance of face parsing a bit as an option of data augmentation.
Non-parametric image synthesis approaches @cite_14 @cite_37 @cite_19 usually generate new images by warping and stitching together existing patches from a database. The idea is extended by Qi al @cite_2 which combines neural networks to improve quality. Though similar at first glance, our method is intrinsically different from non-parametric image synthesis: our local embedding sub-network encodes facial instances as embeddings instead of image patches. New face images in our method are generated through the mask-guided generative sub-network, instead of warping and stitching image patches together. By jointly training all sub-networks, our model generates facial images that are higher quality than non-parametric methods that may also be difficult for them: synthesizing a face with an open mouth showing teeth from a source face with a closed mouth hiding all teeth.
@cite_19 @cite_37 @cite_14 @cite_2
2134921974 1563676183 2171011251
We present a system for inserting new objects into existing photographs by querying a vast image-based object library, pre-computed using a publicly available Internet object database. The central goal is to shield the user from all of the arduous tasks typically involved in image compositing. The user is only asked to do two simple things: 1) pick a 3D location in the scene to place a new object; 2) select an object to insert using a hierarchical menu. We pose the problem of object insertion as a data-driven, 3D-based, context-sensitive object retrieval task. Instead of trying to manipulate the object to change its orientation, color distribution, etc. to fit the new image, we simply retrieve an object of a specified class that has all the required properties (camera pose, lighting, resolution, etc) from our large object library. We present new automatic algorithms for improving object segmentation and blending, estimating true 3D object size and orientation, and estimating scene lighting conditions. We also present an intuitive user interface that makes object insertion fast and simple even for the artistically challenged. Appearance Space Texture Synthesis (ASTS) provides fast texture synthesis by example. Unfortunately, speed is only achieved at the cost of a long pre-processing step. This is tedious for artists and limits the input resolution. In addition, the aggressively pruned search space results in strong artifacts when synthesizing multiple textures under user guidance (Texture-By-Numbers, TBN). In this thesis, we replace the k-coherence search step used by most modern synthesis algorithms with a new parallel, coherent random walk. We show that this drastically improves the synthesis quality with TBN, while maintaining the parallel nature, speed, and flexibility of the original ASTS runtime. Since it removes the expensive pre-computation of k-coherent candidates, we are able to use larger inputs, and start synthesis much faster. This is essential for artists designing high-quality exemplars. What can you do with a million images? In this paper we present a new image completion algorithm powered by a huge database of photographs gathered from the Web. The algorithm patches up holes in images by finding similar image regions in the database that are not only seamless but also semantically valid. Our chief insight is that while the space of images is effectively infinite, the space of semantically differentiable scenes is actually not that large. For many image completion tasks we are able to find similar scenes which contain image fragments that will convincingly complete the image. Our algorithm is entirely data-driven, requiring no annotations or labelling by the user. Unlike existing image completion methods, our algorithm can generate a diverse set of results for each input image and we allow users to select among them. We demonstrate the superiority of our algorithm over existing image completion approaches.