Transfer learning for feature extraction can be used to exploit deep representations in contexts where there is very few training data, where there are limited computational resources, or when tuning the hyper-parameters needed for training is not an option. While previous contributions to feature extraction propose embeddings based on a single layer of the network, in this paper we propose a full-network embedding which successfully integrates convolutional and fully connected features, coming from all layers of a deep convolutional neural network. To do so, the embedding normalizes features in the context of the problem, and discretizes their values to reduce noise and regularize the embedding space. Significantly, this also reduces the computational cost of processing the resultant representations. The proposed method is shown to outperform single layer embeddings on several image classification tasks, while also being more robust to the choice of the pre-trained model used for obtaining the initial features. The performance gap in classification accuracy between thoroughly tuned solutions and the full-network embedding is also reduced, which makes of the proposed approach a competitive solution for a large set of applications. Deep learning models, and particularly convolutional neural networks (CNN), have become the standard approach for tackling image processing tasks. The key to the success of these methods lies in the rich representations deep models build, which are generated after an exhaustive and computationally expensive learning process BID16 . To generate deep representations, deep learning models have strong training requirements in terms of dataset size, computational power and optimal hyper-parametrization. For any domain or application in which either of those factors is an issue, training a deep model from scratch becomes unfeasible.Within deep learning, the field of transfer learning studies how to extract and reuse pre-trained deep representations. This approach has three main applications: improving the performance of a network by initializing its training from a non-random state BID31 BID2 BID17 , enabling the training of deep networks for tasks of limited dataset size BID9 BID27 , and exploiting deep representations through alternative machine learning methods BID0 BID25 BID10 . The first two cases, where training a deep network remains the end purpose of the transfer learning process, are commonly known as transfer learning for fine-tuning, while the third case, where the end purpose of the transfer learning does not necessarily include training a deep net, is typically referred as transfer learning for feature extraction.Of the three limiting factors of training deep networks (i.e., dataset size, computational cost, and optimal hyper-parametrization), transfer learning for fine-tuning partly solves the first. Indeed, one can successfully train a CNN on a dataset composed by roughly a few thousand instances using a pre-trained model as starting point, and achieve state-of-the-art-results. Unfortunately, fine-tuning a model still requires a minimum dataset size, a significant amount of computational resources, and lots of time to optimize the multiple hyper-parameters involved in the process.Transfer learning for feature extraction on the other hand is based on processing a set of data instances through a pre-trained neural network, extracting the activation values so these can be used by another learning mechanism. This is applicable to datasets of any size, as each data instance is processed independently. It has a relatively small computational cost, since there is no deep net training. And finally, it requires no hyper-parameter optimization, since the pre-trained model can be used out-of-the-box. Significantly, the applications of transfer learning for feature extraction are limited only by the capabilities of the methods that one can execute on top of the generated deep representations.As previously mentioned, designing and training a deep model to maximize classification performance is a time consuming task. In this paper we explore the opposite approach, minimizing the design and tuning effort using a feature extraction process. Our goal is to build an out-of-the-box classification tool (which could be used by anyone regardless of technical background) capable of defining a full-network embedding (integrating the representations built by all layers of a source CNN model). When compared to single-layer embeddings, this approach generates richer and more powerful embeddings, while also being more robust to the use of inappropriate pre-trained models. We asses the performance of such solution when compared with thoroughly designed and tuned models. In this paper we describe a feature extraction process which leverages the information encoded in all the features of a deep CNN. The full-network embedding introduces the use of feature standardization and of a novel feature discretization methodology. The former provides context-dependent Table 5 : Classification results in % average per-class accuracy of the baseline and the full-network embedding when using a network pre-trained on ImageNet 2012 for mit67 and on Places2 for the rest. embeddings, which adapt the representations to the problem at hand. The later reduces noise and regularizes the embedding space while keeping the size of the original representation language (i.e., the pre-trained model used as source). Significantly, the feature discretization restricts the computational overhead resultant of processing much larger embeddings when training an SVM. Our experiments also show that the full-network is more robust than single-layer embeddings when an appropriate source model is not available.The resultant full-network embedding is shown to outperform single-layer embeddings in several classification tasks, and to provide the best reported results on one of those tasks (wood). Within the state-of-the-art, the full-network embedding represents the best available solution when one of the following conditions apply: When the accessible data is scarce, or an appropriate pre-trained model is not available (e.g., specialized industrial applications), when computational resources are limited (e.g., no GPUs availability), or when development time or technical expertise is restricted or non cost-effective.Beyond classification, the full-network embedding may be of relevance for any task exploiting visual embeddings. For example, in image retrieval and image annotation tasks, the full-network embedding has been shown to provide a boost in performance when compared to one layer embeddings .
We present a full-network embedding of CNN which outperforms single layer embeddings for transfer learning tasks.
{"src": "scitldr-AIC_train", "paper_id": "SkAK2jg0b"}