Diving Deeper into Underwater Image Enhancement: A Survey <s> DenseGAN <s> We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. The method is straightforward to implement, is computationally efficient, has little memory requirements, is invariant to diagonal rescaling of the gradients, and is well suited for problems that are large in terms of data and/or parameters. The method is also appropriate for non-stationary objectives and problems with very noisy and/or sparse gradients. The hyper-parameters have intuitive interpretations and typically require little tuning. Some connections to related algorithms, on which Adam was inspired, are discussed. We also analyze the theoretical convergence properties of the algorithm and provide a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework. Empirical results demonstrate that Adam works well in practice and compares favorably to other stochastic optimization methods. Finally, we discuss AdaMax, a variant of Adam based on the infinity norm. <s> BIB001 </s> Diving Deeper into Underwater Image Enhancement: A Survey <s> DenseGAN <s> This paper proposes Markovian Generative Adversarial Networks (MGANs), a method for training generative networks for efficient texture synthesis. While deep neural network approaches have recently demonstrated remarkable results in terms of synthesis quality, they still come at considerable computational costs (minutes of run-time for low-res images). Our paper addresses this efficiency issue. Instead of a numerical deconvolution in previous work, we precompute a feed-forward, strided convolutional network that captures the feature statistics of Markovian patches and is able to directly generate outputs of arbitrary dimensions. Such network can directly decode brown noise to realistic texture, or photos to artistic paintings. With adversarial training, we obtain quality comparable to recent neural texture synthesis methods. As no optimization is required at generation time, our run-time performance (0.25 M pixel images at 25 Hz) surpasses previous neural texture synthesizers by a significant margin (at least 500 times faster). We apply this idea to texture synthesis, style transfer, and video stylization. <s> BIB002 </s> Diving Deeper into Underwater Image Enhancement: A Survey <s> DenseGAN <s> One of the challenges in the study of generative adversarial networks is the instability of its training. In this paper, we propose a novel weight normalization technique called spectral normalization to stabilize the training of the discriminator. Our new normalization technique is computationally light and easy to incorporate into existing implementations. We tested the efficacy of spectral normalization on CIFAR10, STL-10, and ILSVRC2012 dataset, and we experimentally confirmed that spectrally normalized GANs (SN-GANs) is capable of generating images of better or equal quality relative to the previous training stabilization techniques. <s> BIB003 </s> Diving Deeper into Underwater Image Enhancement: A Survey <s> DenseGAN <s> Underwater image enhancement has received much attention in underwater vision research. However, raw underwater images easily suffer from color distortion, underexposure, and fuzz caused by the underwater scene. To address the above-mentioned problems, we propose a new multiscale dense generative adversarial network (GAN) for enhancing underwater images. The residual multiscale dense block is presented in the generator, where the multiscale, dense concatenation, and residual learning can boost the performance, render more details, and utilize previous features, respectively. And the discriminator employs computationally light spectral normalization to stabilize the training of the discriminator. Meanwhile, nonsaturating GAN loss function combining $L_1$ loss and gradient loss is presented to focus on image features of ground truth. Final enhanced results on synthetic and real underwater images demonstrate the superiority of the proposed method, which outperforms nondeep and deep learning methods in both qualitative and quantitative evaluations. Furthermore, we perform an ablation study to show the contributions of each component and carry out application tests to further demonstrate the effectiveness of the proposed method. <s> BIB004
To enhance the underwater images, Guo et al. BIB004 introduced a multiscale dense block (MSDB) algorithm, namely, DenseGAN 1 which employs the use of dense connections, residual learning, and multi-scale network for underwater image enhancement. The generator at the start is composed of two convolutional, batch normalization (BN), leaky ReLU (LReLU) sequence then two MSDB blocks followed by sequence Deconvolutional-BN-LReLU, while at the end there is a deconvolutional layer and a TanH layer. The network architecture of the DenseGAN generator and MSDB are shown in Figure 2 . In each MSDB block, the input features are passed through two different branches, where each branch has kernels with different dilations. The features from each branch are concatenated half-way through the MSDB block and fed again into the respective branches. At the end of the MSDB block, the features are concatenated again and passed through a 1×1 convolutional layer. The discriminator network is similar to PatchGAN BIB002 ; however, it is composed of five layers of spectral normalization BIB003 . Except for the first and last layer, the discriminator is composed of sequences of convolutional-BN-LReLU. The first two layers of the generator have 7×7 and 3×3 filter size with 64 and 128 feature maps respectively. The last deconvolution layer outputs the same number of channels as the input. The TanH layer keeps the distribution between -1 and 1. Moreover, the slope of the leaky ReLU is fixed at 0.2, and the network is trained via TensorFlow framework using a learning rate of 10 −3 with patch size of 256×256×3. The ADAM BIB001 is used for optimization, and batch size is set to 32. The losses employed are GAN loss, 1 , and gradient loss.