Deep Learning Models for Retinal Blood Vessels Segmentation: A Review <s> II. OVERVIEW OF DEEP LEARNING METHODS <s> Most of the existing approaches to collaborative filtering cannot handle very large data sets. In this paper we show how a class of two-layer undirected graphical models, called Restricted Boltzmann Machines (RBM's), can be used to model tabular data, such as user's ratings of movies. We present efficient learning and inference procedures for this class of models and demonstrate that RBM's can be successfully applied to the Netflix data set, containing over 100 million user/movie ratings. We also show that RBM's slightly outperform carefully-tuned SVD models. When the predictions of multiple RBM models and multiple SVD models are linearly combined, we achieve an error rate that is well over 6% better than the score of Netflix's own system. <s> BIB001 </s> Deep Learning Models for Retinal Blood Vessels Segmentation: A Review <s> II. OVERVIEW OF DEEP LEARNING METHODS <s> We explore an original strategy for building deep networks, based on stacking layers of denoising autoencoders which are trained locally to denoise corrupted versions of their inputs. The resulting algorithm is a straightforward variation on the stacking of ordinary autoencoders. It is however shown on a benchmark of classification problems to yield significantly lower classification error, thus bridging the performance gap with deep belief networks (DBN), and in several cases surpassing it. Higher level representations learnt in this purely unsupervised fashion also help boost the performance of subsequent SVM classifiers. Qualitative experiments show that, contrary to ordinary autoencoders, denoising autoencoders are able to learn Gabor-like edge detectors from natural image patches and larger stroke detectors from digit images. This work clearly establishes the value of using a denoising criterion as a tractable unsupervised objective to guide the learning of useful higher level representations. <s> BIB002 </s> Deep Learning Models for Retinal Blood Vessels Segmentation: A Review <s> II. OVERVIEW OF DEEP LEARNING METHODS <s> In recent years, deep artificial neural networks (including recurrent ones) have won numerous contests in pattern recognition and machine learning. This historical survey compactly summarizes relevant work, much of it from the previous millennium. Shallow and Deep Learners are distinguished by the depth of their credit assignment paths, which are chains of possibly learnable, causal links between actions and effects. I review deep supervised learning (also recapitulating the history of backpropagation), unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks. <s> BIB003 </s> Deep Learning Models for Retinal Blood Vessels Segmentation: A Review <s> II. OVERVIEW OF DEEP LEARNING METHODS <s> Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors. <s> BIB004 </s> Deep Learning Models for Retinal Blood Vessels Segmentation: A Review <s> II. OVERVIEW OF DEEP LEARNING METHODS <s> This chapter covers successful applications in deep learning with remarkable capability to generate sophisticated and invariant features from raw input signal data. New insights of the visual cortex and studies in the relations between the connectivity found in the brain and mechanisms for mind inference have enlightened the development of deep neural networks. In this chapter the motivation for the design of these architectures points out towards models with many layers exhibiting complex behavior enhanced by thousands of neurons in each layer. By contrast, shallow neural networks have admittedly less capability with respect to inference mechanisms since no feature detectors are found in their hidden layer. Then the chapter formalizes Restricted Boltzmann Machines (RBMs) and Deep Belief Networks (DBNs), which are generative models that along with an unsupervised greedy learning algorithm CD-k are able to attain deep learning of objects. A robust learning adaptive size method is presented. Moreover, a GPU parallel implementation yields high speedups as compared to conventional implementations. Results in benchmark data sets are presented as well as further discussion of these models. <s> BIB005 </s> Deep Learning Models for Retinal Blood Vessels Segmentation: A Review <s> II. OVERVIEW OF DEEP LEARNING METHODS <s> Object detection performance, as measured on the canonical PASCAL VOC Challenge datasets, plateaued in the final years of the competition. The best-performing methods were complex ensemble systems that typically combined multiple low-level image features with high-level context. In this paper, we propose a simple and scalable detection algorithm that improves mean average precision (mAP) by more than 50 percent relative to the previous best result on VOC 2012—achieving a mAP of 62.4 percent. Our approach combines two ideas: (1) one can apply high-capacity convolutional networks (CNNs) to bottom-up region proposals in order to localize and segment objects and (2) when labeled training data are scarce, supervised pre-training for an auxiliary task, followed by domain-specific fine-tuning, boosts performance significantly. Since we combine region proposals with CNNs, we call the resulting model an R-CNN or Region-based Convolutional Network . Source code for the complete system is available at http://www.cs.berkeley.edu/~rbg/rcnn. <s> BIB006
Deep learning is one of machine learning methods. It consists of hierarchical structured layers that can translate input data to meaningful outputs in a black box model. Deep learning methods have wide applications within different research analysis such as graphical modeling of data, neural networks, parameters optimization, image analysis, pattern recognition and signal processing. Many deep learning models in various applications are based on the model, proposed by Yann LeCun for hand written recognition by using deep supervised backpropagation convolutional network BIB004 . In recent six years, deep learning has become a hot research area in the field of computer vision and image processing owing to the achievement in different applications. Krizhevsky et al.  was the first to propose the deep CNN for image classification through ImageNet classification challenge. Along with image classification, object detection and segmentation tasks were also achieved by using CNN models as reported in many research works BIB006 . A rich family of deep learning methods have been developed since 1990 BIB003 , such as Deep Neural Networks (DNN) BIB003 , Auto-encoders(AEs) and Stacked Auto-encoders (SAEs) Neural Network BIB002 , Restricted Boltzmann Machines (RBM) BIB001 , Deep Belief Network (DBN) BIB005 and much more. Among all these studies, deep convolutional neural network demonstrate with better performance on a variety of tasks in images/signal processing and computer vision. In this section, we elaborate the basic concepts of deep from its initial implementation to advance level.