A Survey on Deep Learning Methods for Robot Vision <s> Training <s> In this paper a unified learning framework for object detection and classification using nested cascades of boosted classifiers is proposed. The most interesting aspect of this framework is the integration of powerful learning capabilities together with effective training procedures, which allows building detection and classification systems with high accuracy, robustness, processing speed, and training speed. The proposed framework allows us to build state of the art face detection, eyes detection, and gender classification systems. The performance of these systems is validated and analyzed using standard face databases (BioID, FERET and CMU-MIT), and a new face database (UCHFACE). <s> BIB001 </s> A Survey on Deep Learning Methods for Robot Vision <s> Training <s> Whereas before 2006 it appears that deep multilayer neural networks were not successfully trained, since then several algorithms have been shown to successfully train them, with experimental results showing the superiority of deeper vs less deep architectures. All these experimental results were obtained with new initialization or training mechanisms. Our objective here is to understand better why standard gradient descent from random initialization is doing so poorly with deep neural networks, to better understand these recent relative successes and help design better algorithms in the future. We first observe the influence of the non-linear activations functions. We find that the logistic sigmoid activation is unsuited for deep networks with random initialization because of its mean value, which can drive especially the top hidden layer into saturation. Surprisingly, we find that saturated units can move out of saturation by themselves, albeit slowly, and explaining the plateaus sometimes seen when training neural networks. We find that a new non-linearity that saturates less can often be beneficial. Finally, we study how activations and gradients vary across layers and during training, with the idea that training may be more difficult when the singular values of the Jacobian associated with each layer are far from 1. Based on these considerations, we propose a new initialization scheme that brings substantially faster convergence. 1 Deep Neural Networks Deep learning methods aim at learning feature hierarchies with features from higher levels of the hierarchy formed by the composition of lower level features. They include Appearing in Proceedings of the 13 International Conference on Artificial Intelligence and Statistics (AISTATS) 2010, Chia Laguna Resort, Sardinia, Italy. Volume 9 of JMLR: WC Weston et al., 2008). Much attention has recently been devoted to them (see (Bengio, 2009) for a review), because of their theoretical appeal, inspiration from biology and human cognition, and because of empirical success in vision (Ranzato et al., 2007; Larochelle et al., 2007; Vincent et al., 2008) and natural language processing (NLP) (Collobert & Weston, 2008; Mnih & Hinton, 2009). Theoretical results reviewed and discussed by Bengio (2009), suggest that in order to learn the kind of complicated functions that can represent high-level abstractions (e.g. in vision, language, and other AI-level tasks), one may need deep architectures. Most of the recent experimental results with deep architecture are obtained with models that can be turned into deep supervised neural networks, but with initialization or training schemes different from the classical feedforward neural networks (Rumelhart et al., 1986). Why are these new algorithms working so much better than the standard random initialization and gradient-based optimization of a supervised training criterion? Part of the answer may be found in recent analyses of the effect of unsupervised pretraining (Erhan et al., 2009), showing that it acts as a regularizer that initializes the parameters in a “better” basin of attraction of the optimization procedure, corresponding to an apparent local minimum associated with better generalization. But earlier work (Bengio et al., 2007) had shown that even a purely supervised but greedy layer-wise procedure would give better results. So here instead of focusing on what unsupervised pre-training or semi-supervised criteria bring to deep architectures, we focus on analyzing what may be going wrong with good old (but deep) multilayer neural networks. Our analysis is driven by investigative experiments to monitor activations (watching for saturation of hidden units) and gradients, across layers and across training iterations. We also evaluate the effects on these of choices of activation function (with the idea that it might affect saturation) and initialization procedure (since unsupervised pretraining is a particular form of initialization and it has a drastic impact). <s> BIB002 </s> A Survey on Deep Learning Methods for Robot Vision <s> Training <s> The latest generation of Convolutional Neural Networks (CNN) have achieved impressive results in challenging benchmarks on image recognition and object detection, significantly raising the interest of the community in these methods. Nevertheless, it is still unclear how different CNN methods compare with each other and with previous state-of-the-art shallow representations such as the Bag-of-Visual-Words and the Improved Fisher Vector. This paper conducts a rigorous evaluation of these new techniques, exploring different deep architectures and comparing them on a common ground, identifying and disclosing important implementation details. We identify several useful properties of CNN-based representations, including the fact that the dimensionality of the CNN output layer can be reduced significantly without having an adverse effect on performance. We also identify aspects of deep and shallow methods that can be successfully shared. In particular, we show that the data augmentation techniques commonly applied to CNN-based methods can also be applied to shallow methods, and result in an analogous performance boost. Source code and models to reproduce the experiments in the paper is made publicly available. <s> BIB003 </s> A Survey on Deep Learning Methods for Robot Vision <s> Training <s> The theory of reinforcement learning provides a normative account, deeply rooted in psychological and neuroscientific perspectives on animal behaviour, of how agents may optimize their control of an environment. To use reinforcement learning successfully in situations approaching real-world complexity, however, agents are confronted with a difficult task: they must derive efficient representations of the environment from high-dimensional sensory inputs, and use these to generalize past experience to new situations. Remarkably, humans and other animals seem to solve this problem through a harmonious combination of reinforcement learning and hierarchical sensory processing systems, the former evidenced by a wealth of neural data revealing notable parallels between the phasic signals emitted by dopaminergic neurons and temporal difference reinforcement learning algorithms. While reinforcement learning agents have achieved some successes in a variety of domains, their applicability has previously been limited to domains in which useful features can be handcrafted, or to domains with fully observed, low-dimensional state spaces. Here we use recent advances in training deep neural networks to develop a novel artificial agent, termed a deep Q-network, that can learn successful policies directly from high-dimensional sensory inputs using end-to-end reinforcement learning. We tested this agent on the challenging domain of classic Atari 2600 games. We demonstrate that the deep Q-network agent, receiving only the pixels and the game score as inputs, was able to surpass the performance of all previous algorithms and achieve a level comparable to that of a professional human games tester across a set of 49 games, using the same algorithm, network architecture and hyperparameters. This work bridges the divide between high-dimensional sensory inputs and actions, resulting in the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks. <s> BIB004 </s> A Survey on Deep Learning Methods for Robot Vision <s> Training <s> We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0%, respectively, which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overfitting in the fully connected layers we employed a recently developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry. <s> BIB005 </s> A Survey on Deep Learning Methods for Robot Vision <s> Training <s> We describe a learning-based approach to hand-eye coordination for robotic grasping from monocular images. To learn hand-eye coordination for grasping, we trained a large convolutional neural network to predict the probability that task-space motion of the gripper will result in successful grasps, using only monocular camera images and independently of camera calibration or the current robot pose. This requires the network to observe the spatial relationship between the gripper and objects in the scene, thus learning hand-eye coordination. We then use this network to servo the gripper in real time to achieve successful grasps. To train our network, we collected over 800,000 grasp attempts over the course of two months, using between 6 and 14 robotic manipulators at any given time, with differences in camera placement and hardware. Our experimental evaluation demonstrates that our method achieves effective real-time control, can successfully grasp novel objects, and corrects mistakes by continuous servoing. <s> BIB006
Training, validation and test dataset definitions. The available dataset must be divided into three non-overlapping groups: training, validation, and test. The training dataset is used for learning the model parameters (weights) of the network. The validation dataset is used for selecting the hyper-parameters by evaluating the performance of the network under different hyperparameter configurations. The test dataset is used for characterizing the trained network by estimating its generalization error. Data augmentation is a mechanism used to increase the size of the training datasets by applying some transformations/perturbations to the data in order to generate additional examples. Data augmentation has been used in deep learning (reported, e.g., in BIB005 [74]), but also in traditional object learning methods (e.g. BIB001 ). The most frequently used transformations are geometric ones such as in-plane rotations, flipping, random cropping, and photometric transformations, such as RGB jittering  , color casting, vignetting, and lens distortion  . As shown in BIB003 , the use of data augmentation can increase the performance by about 3%. Weights Initialization. When a network is trained from scratch, a set of initial weights must be specified. Initialization of neural networks is an important topic because, if the initialization is not adequate, the network can converge to bad local minima. Initial weights need to have diversity for breaking symmetries: if all the initial weights are the same, they will converge to the same value per layer, and will not be able to represent the input-output relation. Also, when using sigmoid functions, weights with high values can cause saturation on the neuron's outputs, generating very small gradients that are not useful for learning. A common approach is to use a uniform distribution for weight initialization. For example, a normalized initialization is proposed in BIB002 : where in n is the number of inputs for the neuron, and out n is the number of neurons in the next layer connected to the current one. For the particular case of networks based on ReLU activation function, the following initialization procedure is proposed: Also, when using ReLU units, a small bias near 0.01 can be added to account for the asymmetry of this activation function. Hyper-parameters for the learning process. Once the architecture and initial weights are selected, a tuning of the network to the task dataset must be performed. The learning procedure (gradient descent with momentum) is controlled by several hyperparameters. Recommended hyperparameters for CNNs BIB005 [74] are: momentum 0.9, weight decay 5*10 -4 , and initial learning rate 10 -2 . Learning rate is decreased by a factor of 10 when the validation error stops decreasing. Learning is computationally expensive when compared to final DNN use, and requires large amounts of time, memory, and computing power. In most of the problems, learning can be performed offline in high-end computers, and robotic platforms can be used only for collecting data. Systems based on reinforcement learning, that require interaction of an agent with its environment, have been applied in simulated environments like computer games BIB004 , in which runtime of the full system can be controlled and several trials can be performed in parallel. Furthermore, systems able to learn from interaction with their environment have been developed for some specific tasks like learning eye-hand coordination BIB006 , in which several grippers were used for collecting data, large computer capabilities were available, and supervised learning was adapted for the task. However, online learning on limited hardware platforms is a problem that is far from being solved technically, and is an open research area (see