Diving Deeper into Underwater Image Enhancement: A Survey <s> WaterGAN <s> We address the problem of inferring the pose of an RGB-D camera relative to a known 3D scene, given only a single acquired image. Our approach employs a regression forest that is capable of inferring an estimate of each pixel's correspondence to 3D points in the scene's world coordinate frame. The forest uses only simple depth and RGB pixel comparison features, and does not require the computation of feature descriptors. The forest is trained to be capable of predicting correspondences at any pixel, so no interest point detectors are required. The camera pose is inferred using a robust optimization scheme. This starts with an initial set of hypothesized camera poses, constructed by applying the forest at a small fraction of image pixels. Preemptive RANSAC then iterates sampling more pixels at which to evaluate the forest, counting inliers, and refining the hypothesized poses. We evaluate on several varied scenes captured with an RGB-D camera and observe that the proposed technique achieves highly accurate relocalization and substantially out-performs two state of the art baselines. <s> BIB001 </s> Diving Deeper into Underwater Image Enhancement: A Survey <s> WaterGAN <s> This paper presents an approach for labeling objects in 3D scenes. We introduce HMP3D, a hierarchical sparse coding technique for learning features from 3D point cloud data. HMP3D classifiers are trained using a synthetic dataset of virtual scenes generated using CAD models from an online database. Our scene labeling system combines features learned from raw RGB-D images and 3D point clouds directly, without any hand-designed features, to assign an object label to every 3D point in the scene. Experiments on the RGB-D Scenes Dataset v.2 demonstrate that the proposed approach can be used to label indoor scenes containing both small tabletop objects and large furniture pieces. <s> BIB002 </s> Diving Deeper into Underwater Image Enhancement: A Survey <s> WaterGAN <s> This letter reports on WaterGAN, a generative adversarial network (GAN) for generating realistic underwater images from in-air image and depth pairings in an unsupervised pipeline used for color correction of monocular underwater images. Cameras onboard autonomous and remotely operated vehicles can capture high-resolution images to map the seafloor; however, underwater image formation is subject to the complex process of light propagation through the water column. The raw images retrieved are characteristically different than images taken in air due to effects, such as absorption and scattering, which cause attenuation of light at different rates for different wavelengths. While this physical process is well described theoretically, the model depends on many parameters intrinsic to the water column as well as the structure of the scene. These factors make recovery of these parameters difficult without simplifying assumptions or field calibration; hence, restoration of underwater images is a nontrivial problem. Deep learning has demonstrated great success in modeling complex nonlinear systems but requires a large amount of training data, which is difficult to compile in deep sea environments. Using WaterGAN, we generate a large training dataset of corresponding depth, in-air color images, and realistic underwater images. These data serve as input to a two-stage network for color correction of monocular underwater images. Our proposed pipeline is validated with testing on real data collected from both a pure water test tank and from underwater surveys collected in the field. Source code, sample datasets, and pretrained models are made publicly available. <s> BIB003 </s> Diving Deeper into Underwater Image Enhancement: A Survey <s> WaterGAN <s> We present a novel and practical deep fully convolutional neural network architecture for semantic pixel-wise segmentation termed SegNet. This core trainable segmentation engine consists of an encoder network, a corresponding decoder network followed by a pixel-wise classification layer. The architecture of the encoder network is topologically identical to the 13 convolutional layers in the VGG16 network [1] . The role of the decoder network is to map the low resolution encoder feature maps to full input resolution feature maps for pixel-wise classification. The novelty of SegNet lies is in the manner in which the decoder upsamples its lower resolution input feature map(s). Specifically, the decoder uses pooling indices computed in the max-pooling step of the corresponding encoder to perform non-linear upsampling. This eliminates the need for learning to upsample. The upsampled maps are sparse and are then convolved with trainable filters to produce dense feature maps. We compare our proposed architecture with the widely adopted FCN [2] and also with the well known DeepLab-LargeFOV [3] , DeconvNet [4] architectures. This comparison reveals the memory versus accuracy trade-off involved in achieving good segmentation performance. SegNet was primarily motivated by scene understanding applications. Hence, it is designed to be efficient both in terms of memory and computational time during inference. It is also significantly smaller in the number of trainable parameters than other competing architectures and can be trained end-to-end using stochastic gradient descent. We also performed a controlled benchmark of SegNet and other architectures on both road scenes and SUN RGB-D indoor scene segmentation tasks. These quantitative assessments show that SegNet provides good performance with competitive inference time and most efficient inference memory-wise as compared to other architectures. We also provide a Caffe implementation of SegNet and a web demo at http://mi.eng.cam.ac.uk/projects/segnet. <s> BIB004 </s> Diving Deeper into Underwater Image Enhancement: A Survey <s> WaterGAN <s> Visual 3D reconstruction techniques provide rich ecological and habitat structural information from underwater imagery. However, an unaided swimmer or diver struggles to navigate precisely over larger extents with consistent image overlap needed for visual reconstruction. While underwater robots have demonstrated systematic coverage of areas much larger than the footprint of a single image, access to suitable robotic systems is limited and requires specialized operators. Furthermore, robots are poor at navigating hydrodynamic habitats such as shallow coral reefs. We present a simple approach that constrains the motion of a swimmer using a line unwinding from a fixed central drum. The resulting motion is the involute of a circle, a spiral-like path with constant spacing between revolutions. We test this survey method at a broad range of habitats and hydrodynamic conditions encircling Lizard Island in the Great Barrier Reef, Australia. The approach generates fast, structured, repeatable, and large-extent surveys (~110 m2 in 15 min) that can be performed with two people and are superior to the commonly used “mow the lawn” method. The amount of image overlap is a design parameter, allowing for surveys that can then be reliably used in an automated processing pipeline to generate 3D reconstructions, orthographically projected mosaics, and structural complexity indices. The individual images or full mosaics can also be labeled for benthic diversity and cover estimates. The survey method we present can serve as a standard approach to repeatedly collecting underwater imagery for high-resolution 2D mosaics and 3D reconstructions covering spatial extents much larger than a single image footprint without requiring sophisticated robotic systems or lengthy deployment of visual guides. As such, it opens up cost-effective novel observations to inform studies relating habitat structure to ecological processes and biodiversity at scales and spatial resolutions not readily available previously. <s> BIB005
WaterGAN BIB003 as the name indicates, is a generative adversarial network, which manipulates RGB-D images to simulate underwater images for color correction. The authors present a two-part solution where the first part in the pipeline is the WaterGAN BIB003 , and the second part is the image restoration network, composed of a depth estimation network and a color correction network. The WaterGAN has two systems: a generator G and discriminator D. The generator is a noise vector, which is projected, reshaped and passed through several convolutional and deconvolutional layers which output a synthetic image. The discriminator distinguishes between real image (from another dataset) and synthetic (generated by generator). The generator aims to create images which the discriminator classify as real. The underwater images generated by BIB003 are passed through an image restoration network. The network is inspired by an encoder-decoder architecture, particularly, pixel-wise dense learning, and SegNet BIB004 . The SegNet uses a non-parametric upsampling layer which benefits from the max-pooling index information in the encoder. Furthermore, the authors incorporate the skipping layers in the encoder-decoder architecture to compensate for the high frequencies' loss due to pooling operation. The authors collect 7,000 images from Michigan's Marine Hydrodynamics Laboratory. Another 6,500 images are collected from Port Royal, Jamaica. Similarly, 6,083 images are gathered from the coral reef system, Australia BIB005 . Besides, four Kinect datasets i.e. the B3DO  , the UW RGB-D BIB002 , the NYU  and the Microsoft 7-scenes BIB001 , are utilized to form 15,000 underwater images via WaterGAN, out of which 12,000 are used for training and 3,000 for testing. The depth estimation network is trained separately at a fixed learning rate of 10 −6 while the color correction network is initially trained with an input resolution of 128 × 128 having learning rate 10 −6 . After that, the authors refined the color correction network with input images of 512 × 512 resolution, reducing the base learning rate to 10 −7 . The 2 loss is utilized for depth estimation and color correction networks, and further, as a postprocessing step, the images are normalized i.e. [0,1].