A Survey on the New Generation of Deep Learning in Image Processing <s> 2) DCGAN <s> Unsupervised multilayered (“deep”) models are considered for imagery. The model is represented using a hierarchical convolutional factor-analysis construction, with sparse factor loadings and scores. The computation of layer-dependent model parameters is implemented within a Bayesian setting, employing a Gibbs sampler and variational Bayesian (VB) analysis that explicitly exploit the convolutional nature of the expansion. To address large-scale and streaming data, an online version of VB is also developed. The number of dictionary elements at each layer is inferred from the data, based on a beta-Bernoulli implementation of the Indian buffet process. Example results are presented for several image-processing applications, with comparisons to related models in the literature. <s> BIB001 </s> A Survey on the New Generation of Deep Learning in Image Processing <s> 2) DCGAN <s> In recent years, supervised learning with convolutional networks (CNNs) has seen huge adoption in computer vision applications. Comparatively, unsupervised learning with CNNs has received less attention. In this work we hope to help bridge the gap between the success of CNNs for supervised learning and unsupervised learning. We introduce a class of CNNs called deep convolutional generative adversarial networks (DCGANs), that have certain architectural constraints, and demonstrate that they are a strong candidate for unsupervised learning. Training on various image datasets, we show convincing evidence that our deep convolutional adversarial pair learns a hierarchy of representations from object parts to scenes in both the generator and discriminator. Additionally, we use the learned features for novel tasks - demonstrating their applicability as general image representations. <s> BIB002
A DCGAN firstly introduced convolutional layers to GANs architecture, which can effectively to solve instability of the learning process for the classical GAN (see in Fig.8 ) BIB002 . Generally, there are some architecture guidelines for the stability of DCGAN. The first is the all convolutional networks that replace deterministic spatial pooling functions (such as max pooling) with strided/fractional-strided convolutions and allowing the network to learn its spatial down-sampling or up-sampling. Concretely, pooling layers in discriminator can replace with strided convolutions, and pooling layers in the generator can replace with fractional-strided convolutions. For example, in Fig.9 , we have the following relationship for strided convolutions, output_size = input_size-kernel+2·padding stride + 1 BIB001 where kernel is the size of filters (here, kernel is 3 in Fig.9 ), and ' · ' denotes floor function. Obviously, we have the input x ∈ R 5×5 and the output y ∈ R 2×2 . For fractional-strided convolutions, which can view as the inverse process of strided convolutions. That is, how to implement from y to x ? We need to calculate the new stride and new padding for expanding y, and then we have the following relationship for fractionalstrided convolutions, where input (new) is the size of y. More cases can be referred to the relevant literature. The second is BN, which stabilizes learning by normalizing the input to each unit to have zero mean and unit variance. BN can helps to deal with training problems that arise due to poor initialization and maintains gradient flow in deeper models. Besides, BN can use in other layers both of the generator and the discriminator rather than the generator output layer and the discriminator input layer. The third is that remove fully connected hidden layers for deeper architectures. The fourth is that the ReLU activation can use in the generator except for the output layer which uses the Tanh function, and the Leaky_ReLU activation function, is used in the discriminator for all layers, where α is a small constant, it means that the negative axis information will not be completely lost.