A Survey on the New Generation of Deep Learning in Image Processing <s> F. SEMANTIC SEGMENTATION <s> There is large consent that successful training of deep networks requires many thousand annotated training samples. In this paper, we present a network and training strategy that relies on the strong use of data augmentation to use the available annotated samples more efficiently. The architecture consists of a contracting path to capture context and a symmetric expanding path that enables precise localization. We show that such a network can be trained end-to-end from very few images and outperforms the prior best method (a sliding-window convolutional network) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks. Using the same network trained on transmitted light microscopy images (phase contrast and DIC) we won the ISBI cell tracking challenge 2015 in these categories by a large margin. Moreover, the network is fast. Segmentation of a 512x512 image takes less than a second on a recent GPU. The full implementation (based on Caffe) and the trained networks are available at http://lmb.informatik.uni-freiburg.de/people/ronneber/u-net . <s> BIB001 </s> A Survey on the New Generation of Deep Learning in Image Processing <s> F. SEMANTIC SEGMENTATION <s> Scene parsing is challenging for unrestricted open vocabulary and diverse scenes. In this paper, we exploit the capability of global context information by different-region-based context aggregation through our pyramid pooling module together with the proposed pyramid scene parsing network (PSPNet). Our global prior representation is effective to produce good quality results on the scene parsing task, while PSPNet provides a superior framework for pixel-level prediction. The proposed approach achieves state-of-the-art performance on various datasets. It came first in ImageNet scene parsing challenge 2016, PASCAL VOC 2012 benchmark and Cityscapes benchmark. A single PSPNet yields the new record of mIoU accuracy 85.4% on PASCAL VOC 2012 and accuracy 80.2% on Cityscapes. <s> BIB002
Semantic segmentation understands an image at the pixel level, i.e., assign each pixel in the image an object class. For the semantic segmentation, one of the popular initial deep learning approaches was patch classification, where each pixel was separately classified using a patch of the image around it. However, the computational efficiency of this patch-wise training is too low. Furthermore, the required fixed size patch images are also very hindrance. To address these issues, FCN allows segmentation maps to be generated for the image of any size and is also much faster compared to the patch classification approach. The main advantages of FCN framework are that replace fully connected layers with convolutional layer and implement end-to-end training. Almost all the following state of the art approaches on semantic segmentation adopted this design method. Next, we introduce the following two representative improvements on FCN architecture. First, apart from fully connected layers, one of the main problems with using CNN for segmentation is pooling layers, which can make the where information was discarding. To tackle this issue, SegNet is designed to be an efficient encoder-decoder architecture for pixel-wise semantic segmentation  . In Fig.30 , for the encoder part, we can discard the fully connected layers in favor of retaining higher resolution feature maps at the deepest encoder output. In addition, each encoder layer has a corresponding decoder layer. Then the final output of SegNet is fed to a softmax classifier to produce class probabilities for each pixel independently. Notably, the main advantage for SegNet is using pooling indices transferred to the decoder to improve the segmentation. Second, the major disadvantage of FCN is a lack of suitable strategy to utilize global scene category clues. To incorporate appropriate comprehensive features, spatial pyramid pooling can embed in the FCN-based network, the obtained PSPNet BIB002 can capture both local and global context information to make the final prediction more reliable. Most deep networks architecture on semantic segmentation use natural image datasets can not directly applicable to biomedical images. The U-Net can achieve outstanding performance on biomedical image segmentation, which can view as the extent of FCN architecture. Notably, the main idea of U-Net is to supplement a usual contracting network by successive layers, where pooling operators can replace by upsampling operators. MOreover, the main advantage of U-Net is that it works with very few training images and yields more precise segmentation BIB001 . Meanwhile, U-Net applies to various biomedical segmentation tasks. In TABLE 6, We also summarize some benchmark datasets and corresponding deep learning methods commonly used in semantic segmentation. Compared with the classical models, these new generation models are also intended to highlight novel framework or architecture design and relatively excellent generalization performance.