Image-to-image translation has been made much progress with embracing Generative Adversarial Networks (GANs). However, it's still very challenging for translation tasks that require high-quality, especially at high-resolution and photo-reality. In this paper, we present Discriminative Region Proposal Adversarial Networks (DRPANs) with three components: a generator, a discriminator and a reviser, for high-quality image-to-image translation. To reduce the artifacts and blur problems while translation, based on GAN, we explore a patch discriminator to find and extract the most artificial image patch by sliding the output score map with a window, and map the proposed image patch onto the synthesized fake image as our discriminative region. We then mask the corresponding real image using the discriminative region to obtain a fake-mask real image. For providing constructive revisions to generator, we propose a reviser for GANs to distinguish the real from the fake-mask real for producing realistic details and serve as auxiliaries to generate high-quality translation results. Experiments on a variety of image-to-image translation tasks and datasets validate that our method outperforms state-of-the-art translation methods for producing high-quality translation results in terms of both human perceptual studies and automatic quantitative measures.
Deep Convolutional Neural Networks (CNNs) have been performed well on many computer vision tasks. For style transform problems @cite_20 , many studies were mainly based on VGG-16 network architecture @cite_31 and used perceptual losses for style translation @cite_34 . Network architectures that work well on object recognition tasks have been proved to work well on generative models, , some computer vision translation and editing tasks used residual block as a strong feature learning representation architecture @cite_23 @cite_32 . Feed-forward CNNs accompanied with per-pixel loss have been presented for image super-resolution @cite_33 @cite_3 @cite_36 @cite_20 , image colorization @cite_10 @cite_28 , and semantic segmentation @cite_11 @cite_0 @cite_17 . A recent work for photo realistic image synthesis system, called CRN @cite_30 , can synthesize images with high resolution. However, the images synthesized by feed-forward based approach usually become smooth too much rather than realistic, , not sharp enough in details. Besides, these methods are limited to be applied to other image-to-image translation tasks.
@cite_30 @cite_33 @cite_36 @cite_28 @cite_32 @cite_3 @cite_17 @cite_0 @cite_23 @cite_31 @cite_34 @cite_10 @cite_20 @cite_11
2807007689 2509155366 2415731916 2953264111
Deep convolutional network architectures are often assumed to guarantee generalization for small image translations and deformations. In this paper we show that modern CNNs (VGG16, ResNet50, and InceptionResNetV2) can drastically change their output when an image is translated in the image plane by a few pixels, and that this failure of generalization also happens with other realistic small image transformations. Furthermore, the deeper the network the more we see these failures to generalize. We show that these failures are related to the fact that the architecture of modern CNNs ignores the classical sampling theorem so that generalization is not guaranteed. We also show that biases in the statistics of commonly used image datasets makes it unlikely that CNNs will learn to be invariant to these transformations. Taken together our results suggest that the performance of CNNs in object recognition falls far short of the generalization capabilities of humans. Automatically detecting illustrations is needed for the target system.Deep Convolutional Neural Networks have been successful in computer vision tasks.DCNN with fine-tuning outperformed the other models including handcrafted features. Systems for aggregating illustrations require a function for automatically distinguishing illustrations from photographs as they crawl the network to collect images. A previous attempt to implement this functionality by designing basic features that were deemed useful for classification achieved an accuracy of only about 58 . On the other hand, deep neural networks had been successful in computer vision tasks, and convolutional neural networks (CNNs) had performed good at extracting such useful image features automatically. We evaluated alternative methods to implement this classification functionality with focus on deep neural networks. As the result of experiments, the method that fine-tuned deep convolutional neural network (DCNN) acquired 96.8 accuracy, outperforming the other models including the custom CNN models that were trained from scratch. We conclude that DCNN with fine-tuning is the best method for implementing a function for automatically distinguishing illustrations from photographs. Deep convolutional neural networks (CNNs) have been immensely successful in many high-level computer vision tasks given large labelled datasets. However, for video semantic object segmentation, a domain where labels are scarce, effectively exploiting the representation power of CNN with limited training data remains a challenge. Simply borrowing the existing pre-trained CNN image recognition model for video segmentation task can severely hurt performance. We propose a semi-supervised approach to adapting CNN image recognition model trained from labelled image data to the target domain exploiting both semantic evidence learned from CNN, and the intrinsic structures of video data. By explicitly modelling and compensating for the domain shift from the source domain to the target domain, this proposed approach underpins a robust semantic object segmentation method against the changes in appearance, shape and occlusion in natural videos. We present extensive experiments on challenging datasets that demonstrate the superior performance of our approach compared with the state-of-the-art methods. Deep convolutional neural networks (CNNs) have been immensely successful in many high-level computer vision tasks given large labeled datasets. However, for video semantic object segmentation, a domain where labels are scarce, effectively exploiting the representation power of CNN with limited training data remains a challenge. Simply borrowing the existing pretrained CNN image recognition model for video segmentation task can severely hurt performance. We propose a semi-supervised approach to adapting CNN image recognition model trained from labeled image data to the target domain exploiting both semantic evidence learned from CNN, and the intrinsic structures of video data. By explicitly modeling and compensating for the domain shift from the source domain to the target domain, this proposed approach underpins a robust semantic object segmentation method against the changes in appearance, shape and occlusion in natural videos. We present extensive experiments on challenging datasets that demonstrate the superior performance of our approach compared with the state-of-the-art methods.