A Survey on Deep Learning Methods for Robot Vision <s> Discussion: Current and Future Challenges of Deep Learning in Robotic Applications <s> Image classification is a critical task for both humans and computers. One of the challenges lies in the large scale of the semantic space. In particular, humans can recognize tens of thousands of object classes and scenes. No computer vision algorithm today has been tested at this scale. This paper presents a study of large scale categorization including a series of challenging experiments on classification with more than 10, 000 image classes. We find that a) computational issues become crucial in algorithm design; b) conventional wisdom from a couple of hundred image categories on relative performance of different classifiers does not necessarily hold when the number of categories increases; c) there is a surprisingly strong relationship between the structure of WordNet (developed for studying language) and the difficulty of visual categorization; d) classification can be improved by exploiting the semantic hierarchy. Toward the future goal of developing automatic vision algorithms to recognize tens of thousands or even millions of image categories, we make a series of observations and arguments about dataset scale, category density, and image hierarchy. <s> BIB001 </s> A Survey on Deep Learning Methods for Robot Vision <s> Discussion: Current and Future Challenges of Deep Learning in Robotic Applications <s> Neural networks are both computationally intensive and memory intensive, making them difficult to deploy on embedded systems with limited hardware resources. To address this limitation, we introduce "deep compression", a three stage pipeline: pruning, trained quantization and Huffman coding, that work together to reduce the storage requirement of neural networks by 35x to 49x without affecting their accuracy. Our method first prunes the network by learning only the important connections. Next, we quantize the weights to enforce weight sharing, finally, we apply Huffman coding. After the first two steps we retrain the network to fine tune the remaining connections and the quantized centroids. Pruning, reduces the number of connections by 9x to 13x; Quantization then reduces the number of bits that represent each connection from 32 to 5. On the ImageNet dataset, our method reduced the storage required by AlexNet by 35x, from 240MB to 6.9MB, without loss of accuracy. Our method reduced the size of VGG-16 by 49x from 552MB to 11.3MB, again with no loss of accuracy. This allows fitting the model into on-chip SRAM cache rather than off-chip DRAM memory. Our compression method also facilitates the use of complex neural networks in mobile applications where application size and download bandwidth are constrained. Benchmarked on CPU, GPU and mobile GPU, compressed network has 3x to 4x layerwise speedup and 3x to 7x better energy efficiency. <s> BIB002 </s> A Survey on Deep Learning Methods for Robot Vision <s> Discussion: Current and Future Challenges of Deep Learning in Robotic Applications <s> Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors. <s> BIB003 </s> A Survey on Deep Learning Methods for Robot Vision <s> Discussion: Current and Future Challenges of Deep Learning in Robotic Applications <s> The theory of reinforcement learning provides a normative account, deeply rooted in psychological and neuroscientific perspectives on animal behaviour, of how agents may optimize their control of an environment. To use reinforcement learning successfully in situations approaching real-world complexity, however, agents are confronted with a difficult task: they must derive efficient representations of the environment from high-dimensional sensory inputs, and use these to generalize past experience to new situations. Remarkably, humans and other animals seem to solve this problem through a harmonious combination of reinforcement learning and hierarchical sensory processing systems, the former evidenced by a wealth of neural data revealing notable parallels between the phasic signals emitted by dopaminergic neurons and temporal difference reinforcement learning algorithms. While reinforcement learning agents have achieved some successes in a variety of domains, their applicability has previously been limited to domains in which useful features can be handcrafted, or to domains with fully observed, low-dimensional state spaces. Here we use recent advances in training deep neural networks to develop a novel artificial agent, termed a deep Q-network, that can learn successful policies directly from high-dimensional sensory inputs using end-to-end reinforcement learning. We tested this agent on the challenging domain of classic Atari 2600 games. We demonstrate that the deep Q-network agent, receiving only the pixels and the game score as inputs, was able to surpass the performance of all previous algorithms and achieve a level comparable to that of a professional human games tester across a set of 49 games, using the same algorithm, network architecture and hyperparameters. This work bridges the divide between high-dimensional sensory inputs and actions, resulting in the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks. <s> BIB004 </s> A Survey on Deep Learning Methods for Robot Vision <s> Discussion: Current and Future Challenges of Deep Learning in Robotic Applications <s> We introduce gvnn, a neural network library in Torch aimed towards bridging the gap between classic geometric computer vision and deep learning. Inspired by the recent success of Spatial Transformer Networks, we propose several new layers which are often used as parametric transformations on the data in geometric computer vision. These layers can be inserted within a neural network much in the spirit of the original spatial transformers and allow backpropagation to enable end-to-end learning of a network involving any domain knowledge in geometric computer vision. This opens up applications in learning invariance to 3D geometric transformation for place recognition, end-to-end visual odometry, depth estimation and unsupervised learning through warping with a parametric transformation for image reconstruction error. <s> BIB005 </s> A Survey on Deep Learning Methods for Robot Vision <s> Discussion: Current and Future Challenges of Deep Learning in Robotic Applications <s> The game of Go has long been viewed as the most challenging of classic games for artificial intelligence owing to its enormous search space and the difficulty of evaluating board positions and moves. Here we introduce a new approach to computer Go that uses ‘value networks’ to evaluate board positions and ‘policy networks’ to select moves. These deep neural networks are trained by a novel combination of supervised learning from human expert games, and reinforcement learning from games of self-play. Without any lookahead search, the neural networks play Go at the level of stateof-the-art Monte Carlo tree search programs that simulate thousands of random games of self-play. We also introduce a new search algorithm that combines Monte Carlo simulation with value and policy networks. Using this search algorithm, our program AlphaGo achieved a 99.8% winning rate against other Go programs, and defeated the human European Go champion by 5 games to 0. This is the first time that a computer program has defeated a human professional player in the full-sized game of Go, a feat previously thought to be at least a decade away. <s> BIB006 </s> A Survey on Deep Learning Methods for Robot Vision <s> Discussion: Current and Future Challenges of Deep Learning in Robotic Applications <s> Reinforcement learning provides a powerful and flexible framework for automated acquisition of robotic motion skills. However, applying reinforcement learning requires a sufficiently detailed representation of the state, including the configuration of task-relevant objects. We present an approach that automates state-space construction by learning a state representation directly from camera images. Our method uses a deep spatial autoencoder to acquire a set of feature points that describe the environment for the current task, such as the positions of objects, and then learns a motion skill with these feature points using an efficient reinforcement learning method based on local linear models. The resulting controller reacts continuously to the learned feature points, allowing the robot to dynamically manipulate objects in the world with closed-loop control. We demonstrate our method with a PR2 robot on tasks that include pushing a free-standing toy block, picking up a bag of rice using a spatula, and hanging a loop of rope on a hook at various positions. In each task, our method automatically learns to track task-relevant objects and manipulate their configuration with the robot's arm. <s> BIB007 </s> A Survey on Deep Learning Methods for Robot Vision <s> Discussion: Current and Future Challenges of Deep Learning in Robotic Applications <s> Learning to navigate in complex environments with dynamic elements is an important milestone in developing AI agents. In this work we formulate the navigation question as a reinforcement learning problem and show that data efficiency and task performance can be dramatically improved by relying on additional auxiliary tasks to bootstrap learning. In particular we consider jointly learning the goal-driven reinforcement learning problem with an unsupervised depth prediction task and a self-supervised loop closure classification task. Using this approach we can learn to navigate from raw sensory input in complicated 3D mazes, approaching human-level performance even under conditions where the goal location changes frequently. We provide detailed analysis of the agent behaviour, its ability to localise, and its network activity dynamics, that show that the agent implicitly learns key navigation abilities, with only sparse rewards and without direct supervision. <s> BIB008 </s> A Survey on Deep Learning Methods for Robot Vision <s> Discussion: Current and Future Challenges of Deep Learning in Robotic Applications <s> The main goal of this paper is to analyze the general problem of using Convolutional Neural Networks (CNNs) in robots with limited computational capabilities, and to propose general design guidelines for their use. In addition, two different CNN based NAO robot detectors that are able to run in real-time while playing soccer are proposed. One of the detectors is based on the XNOR-Net and the other on the SqueezeNet. Each detector is able to process a robot object-proposal in ~1ms, with an average number of 1.5 proposals per frame obtained by the upper camera of the NAO. The obtained detection rate is ~97%. <s> BIB009 </s> A Survey on Deep Learning Methods for Robot Vision <s> Discussion: Current and Future Challenges of Deep Learning in Robotic Applications <s> Two less addressed issues of deep reinforcement learning are (1) lack of generalization capability to new target goals, and (2) data inefficiency i.e., the model requires several (and often costly) episodes of trial and error to converge, which makes it impractical to be applied to real-world scenarios. In this paper, we address these two issues and apply our model to the task of target-driven visual navigation. To address the first issue, we propose an actor-critic model whose policy is a function of the goal as well as the current state, which allows to better generalize. To address the second issue, we propose AI2-THOR framework, which provides an environment with high-quality 3D scenes and physics engine. Our framework enables agents to take actions and interact with objects. Hence, we can collect a huge number of training samples efficiently. ::: We show that our proposed method (1) converges faster than the state-of-the-art deep reinforcement learning methods, (2) generalizes across targets and across scenes, (3) generalizes to a real robot scenario with a small amount of fine-tuning (although the model is trained in simulation), (4) is end-to-end trainable and does not need feature engineering, feature matching between frames or 3D reconstruction of the environment. ::: The supplementary video can be accessed at the following link: this https URL <s> BIB010
As already mentioned, the increase in the use of deep learning in robotics will depend on the possibility that DNN methods will be able to adapt to the requirements of robotics applications, such as real-time operation with limited on-board computational resources, and the ability to deal with constrained observational conditions derived from the robot geometry, limited camera resolution, and sensor/object relative pose. The current requirements of large computational power and memory needed for most CNN models used in vision applications are relevant barriers to its adoption in resource-constrained robotic applications such as UAVs, humanoid biped robots, or autonomous vehicles. Efforts are already being made to develop CNN architectures that can be compressed and quantized for use in resource-constrained platforms BIB002 [90] (see Section 2.8). For instance, in BIB009 a CNN based robot detector is presented that is able to run in real-time, in NAO robots while playing soccer. In addition, companies such as Intel, NVIDIA, and Samsung, just to name a few, are developing CNN chips that will enable real-time vision applications BIB003 . For instance, mobile GPU processors like NVIDIA Tegra K1 enable efficient implementation of deep learning algorithms with low power consumption, which is relevant for mobile robotics  . It is expected that these methodologies will consolidate in the next few years, and will then be available to the developers of robot vision applications. The ability of deep learning models to manage spatial invariance to the input data in a computationally and efficient manner is another important issue. In order to address this, the Spatial Transformer Module (STM) was recently introduced  . This corresponds to a self-contained module that can be incorporated into DNNs, and that is able to perform explicit spatial transformation of the features. The most important characteristic of STM is that its parameters (i.e. the transformations of the spatial transformation) can be learned together with the other parameters of the network using the back propagation of the loss. The further development of the STM concept will allow addressing the required invariance to the observational conditions derived from the robot geometry, limited camera resolution, and sensor/object relative pose. STMs are able to deal with rotations, translations, and scaling  , and STMs are already being extended to deal with 3D transformations BIB005 . Further development of STM-inspired techniques is expected in the next few years. Unsupervised learning is another relevant area of future development for deep-learning based vision systems. Biological learning is largely unsupervised; animals and humans discover the structure of the world by exploring and observing it. Therefore, one would expect that similar mechanisms could be used in robotics and other computer-based systems. Until now, the success of purely supervised learning, which is based largely on the availability of massive labeled data, has overshadowed the application of unsupervised learning in the development of vision systems. However, when considering the increasing availability of non-labeled digital data, and the increasing number of vision applications that need to be addressed, one would expect further development of unsupervised learning strategies for DNN models BIB003  . In the case of robotics applications, a natural way of addressing this issue is by using deep learning and reinforcement learning together. This strategy has already been applied for the learning of games (Atari, Go, etc.) BIB004 BIB006 , for solving simple tasks like pushing a box BIB007 , and for solving more complex tasks such as navigating on simulated environments BIB010 BIB008 , but not for robots learning in the wild. The visual system of most animals is active, i.e. animals decide where to look, based on a combination of internal and external stimuli. In a robotics system having a similar active vision approach, reinforcement learning can be used for deciding where to look according to the results of the interaction of the robot with the environment. A related relevant issue is that of open-world learning  , i.e. to learn to detect new classes incrementally, or to learn to distinguish among subclasses incrementally after the "main" one has been learned. If this can be done without supervision, new classifiers can be built based on those that already exist, greatly reducing the effort required to learn new object classes. Note that humans are continuously inventing new objects, fashion changes, etc., and therefore robot vision systems will need to be updated continuously, adding new classes, and/or updating existing ones  . Some recent work has addressed these issues, based mainly on the joint use of deep-learning and transfer-learning methods  [183]. Classification using a very large number of classes is another important challenge to address. AlexNet, like most CNN networks, was developed for solving problems which contain ~1,000 classes (e.g., ILSVRC 2012 challenge). It uses fully connected units and a final softmax unit for classification. However, when working on problems with a very large number of classes (10,000+) in which dense data sampling is available, nearest neighbors could work better than other classifiers BIB001 . Also, the use of a hierarchy-aware cost function (based on WordNet) could enable providing more detailed information when the classes have redundancy BIB001 . However, in current object recognition benchmarks (like ILSVRC 2016), the number of classes has not increased, and no hierarchical information has been used. The ability of learning systems to deal with a high number of classes is an important challenge that needs to be addressed for performing open-world learning. Another important area of research is the combination of new deep learning based methods with classical vision methods based on geometry, which has been very successful in robotics (e.g. visual SLAM). This topic has been addressed at recent workshops  and conferences [59] . There are several ways in which these important and complementary paradigms can be combined. On the one hand, geometry-based methods such as Structure from Motion and Visual SLAM can be used for the training of deeplearning based vision systems; geometry-based methods can extract and model the structure of the environment, and this structure can be used for assisting the learning process of a DNN. On the other hand, DNNs can also be used for learning to compute the visual odometry automatically, and eventually to learn the parameters of a visual SLAM system. In a very recent study, STMs were extended to 3D spatial transformations BIB005 , allowing end-to-end learning of DNNs for computing visual odometry. It is expected that STM-based techniques will allow end-to-end training for optical flow, depth estimation, place recognition with geometric invariance, small-scale bundle adjustment, etc. BIB005 . A final relevant research topic is the direct analysis of video sequences in robotics applications. The analysis of video sequences by using current deep-learning based techniques is very demanding of computational resources. It is expected that the increase of computing capabilities will enable the analysis of video sequences in real-time in the near future. Also, the availability of labeled video datasets for learning tasks such as action recognition will enable the improvement of robot perception tasks, which are currently addressed by using independent CNNs for individual video frames.