A Survey on Deep Learning Methods for Robot Vision <s> Developing Tools: Learning Frameworks and Databases <s> Pylearn2 is a machine learning research library. This does not just mean that it is a collection of machine learning algorithms that share a common API; it means that it has been designed for flexibility and extensibility in order to facilitate research projects that involve new or unusual use cases. In this paper we give a brief history of the library, an overview of its basic philosophy, a summary of the library's architecture, and a description of how the Pylearn2 community functions socially. <s> BIB001 </s> A Survey on Deep Learning Methods for Robot Vision <s> Developing Tools: Learning Frameworks and Databases <s> We introduce two Python frameworks to train neural networks on large datasets: Blocks and Fuel. Blocks is based on Theano, a linear algebra compiler with CUDA-support (Bastien et al., 2012; Bergstra et al., 2010). It facilitates the training of complex neural network models by providing parametrized Theano operations, attaching metadata to Theano’s symbolic computational graph, and providing an extensive set of utilities to assist training the networks, e.g. training algorithms, logging, monitoring, visualization, and serialization. Fuel provides a standard format for machine learning datasets. It allows the user to easily iterate over large datasets, performing many types of pre-processing on the fly. <s> BIB002 </s> A Survey on Deep Learning Methods for Robot Vision <s> Developing Tools: Learning Frameworks and Databases <s> MXNet is a multi-language machine learning (ML) library to ease the development of ML algorithms, especially for deep neural networks. Embedded in the host language, it blends declarative symbolic expression with imperative tensor computation. It offers auto differentiation to derive gradients. MXNet is computation and memory efficient and runs on various heterogeneous systems, ranging from mobile devices to distributed GPU clusters. ::: This paper describes both the API design and the system implementation of MXNet, and explains how embedding of both symbolic expression and tensor operation is handled in a unified fashion. Our preliminary experiments reveal promising results on large scale deep neural network applications using multiple GPU machines. <s> BIB003 </s> A Survey on Deep Learning Methods for Robot Vision <s> Developing Tools: Learning Frameworks and Databases <s> Theano is a Python library that allows to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently. Since its introduction, it has been one of the most used CPU and GPU mathematical compilers - especially in the machine learning community - and has shown steady performance improvements. Theano is being actively and continuously developed since 2008, multiple frameworks have been built on top of it and it has been used to produce many state-of-the-art machine learning models. ::: The present article is structured as follows. Section I provides an overview of the Theano software and its community. Section II presents the principal features of Theano and how to use them, and compares them with other similar projects. Section III focuses on recently-introduced functionalities and improvements. Section IV compares the performance of Theano against Torch7 and TensorFlow on several machine learning models. Section V discusses current limitations of Theano and potential ways of improving it. <s> BIB004
Given the complexity in the design and training of DNNs, special learning frameworks/tools are used for these tasks. There is a wide variety of frameworks available for training and using deep neural networks (see a list in [164]), as well as several public databases suited to various applications and used for training (see a list in [165]). Table 3 presents some of these frameworks, showing the DNN models included in each case. Some of the most popular learning frameworks are described in the following paragraphs. These frameworks were selected by considering the availability of pre-trained models, training/testing speed, and specific advantages such as the inclusion of RBMs in Torch, or multi GPU optimization in CNTK. Caffe  is a deep learning framework written in C++, able to use CUDA [152] and CUDNN [153] for GPU computing. It can also be used on systems without GPUs. Caffe is designed to be used in computer vision applications. The framework can be used through Python and MATLAB, and it has a model zoo [178] , which is a collection of several state-of-the-art pre-trained models for several computer vision tasks. Pre-trained models from the Caffe Zoo can be downloaded and used in a straightforward way. The model zoo includes AlexNet, CaffeNet (an AlexNet variant used across tutorials), R-CNN, GoogleNet, VGG, and SegNet, among other pre-trained models. Caffe has several dependencies, and its installation process is not straightforward. Torch [130] is a scientific computing framework with wide support for machine learning. It is written in C, and able to use CUDA [152] and CUDNN [153] . It can be accessed by using a scripting language called LuaJIT. As it is a general machine learning framework, it is able to train LSTMs and energy-based models such as DBMs. It is also able to load some models from the Caffe Model Zoo. It is embeddable, with ports to iOS, Android, and FPGA backends. Theano BIB004 [132] is a deep learning framework written in Python, that is able to use CUDA [152] and CUDNN [153] . It can run as fast as C frameworks with large datasets and GPUs. It enables the use of symbolic differentiation, which speeds development. It can also generate customized C code for many mathematical operations and its installation procedure is straightforward. There are other frameworks built on Theano, such as Pylearn2 BIB001 , Theano Lights  , Blocks BIB002 and Lasagne  . Pylearn includes algorithms such as CNNs, LSTMs, and auto-encoders. Theano is able to read Caffe models by using Lasagne. Tensorflow [137] is an open source library for numerical computation using data flow graphs. Nodes represent operations, while edges represent multidimensional data arrays (tensors). Data flow graphs can be shown using a graphical interface that enables monitoring the system in real-time. It is able to make computations using several CPUs and GPUs, by using CUDA [152] and CUDNN [153] . The graphs can be implemented in Python and C++. Also, Tensorflow is able to generate stand-alone code for use of feed-forward DNNs, CNNs, and LSTMs. CNTK internal implementation is based on C++. It is able to make computations using several CPUs and GPUs, by using CUDA [152] and CUDNN [153] . CNTK is able to outperform other frameworks in training speed when multiple GPUs are used. For the designer, it is very important to select the framework best suited to the needs and resources of the final application. In It has examples of small CNNs and RNN with LSTM in its tutorial. Can load models from Caffe Zoo. MXNet BIB003 Deep learning framework. It is portable, allows multi-GPU/multi-machine use, and has bindings for Python, R, C++ and Julia. Includes three pre-trained models: Inception-BN Network, Inception-V3 Network, and Full ImageNet Network.