Deep Learning Algorithms with Applications to Video Analytics for A Smart City: A Survey <s> IV. FACE RECOGNITION <s> This paper evaluates the performance both of some texture measures which have been successfully used in various applications and of some new promising approaches proposed recently. For classification a method based on Kullback discrimination of sample and prototype distributions is used. The classification results for single features with one-dimensional feature value distributions and for pairs of complementary features with two-dimensional distributions are presented <s> BIB001 </s> Deep Learning Algorithms with Applications to Video Analytics for A Smart City: A Survey <s> IV. FACE RECOGNITION <s> Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day. <s> BIB002 </s> Deep Learning Algorithms with Applications to Video Analytics for A Smart City: A Survey <s> IV. FACE RECOGNITION <s> Most modern face recognition systems rely on a feature representation given by a hand-crafted image descriptor, such as Local Binary Patterns (LBP), and achieve improved performance by combining several such representations. In this paper, we propose deep learning as a natural source for obtaining additional, complementary representations. To learn features in high-resolution images, we make use of convolutional deep belief networks. Moreover, to take advantage of global structure in an object class, we develop local convolutional restricted Boltzmann machines, a novel convolutional learning model that exploits the global structure by not assuming stationarity of features across the image, while maintaining scalability and robustness to small misalignments. We also present a novel application of deep learning to descriptors other than pixel intensity values, such as LBP. In addition, we compare performance of networks trained using unsupervised learning against networks with random filters, and empirically show that learning weights not only is necessary for obtaining good multilayer representations, but also provides robustness to the choice of the network architecture parameters. Finally, we show that a recognition system using only representations obtained from deep learning can achieve comparable accuracy with a system using a combination of hand-crafted image descriptors. Moreover, by combining these representations, we achieve state-of-the-art results on a real-world face verification database. <s> BIB003 </s> Deep Learning Algorithms with Applications to Video Analytics for A Smart City: A Survey <s> IV. FACE RECOGNITION <s> We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0% which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overriding in the fully-connected layers we employed a recently-developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry. <s> BIB004 </s> Deep Learning Algorithms with Applications to Video Analytics for A Smart City: A Survey <s> IV. FACE RECOGNITION <s> In modern face recognition, the conventional pipeline consists of four stages: detect => align => represent => classify. We revisit both the alignment step and the representation step by employing explicit 3D face modeling in order to apply a piecewise affine transformation, and derive a face representation from a nine-layer deep neural network. This deep network involves more than 120 million parameters using several locally connected layers without weight sharing, rather than the standard convolutional layers. Thus we trained it on the largest facial dataset to-date, an identity labeled dataset of four million facial images belonging to more than 4, 000 identities. The learned representations coupling the accurate model-based alignment with the large facial database generalize remarkably well to faces in unconstrained environments, even with a simple classifier. Our method reaches an accuracy of 97.35% on the Labeled Faces in the Wild (LFW) dataset, reducing the error of the current state of the art by more than 27%, closely approaching human-level performance. <s> BIB005 </s> Deep Learning Algorithms with Applications to Video Analytics for A Smart City: A Survey <s> IV. FACE RECOGNITION <s> This paper proposes to learn a set of high-level feature representations through deep learning, referred to as Deep hidden IDentity features (DeepID), for face verification. We argue that DeepID can be effectively learned through challenging multi-class face identification tasks, whilst they can be generalized to other tasks (such as verification) and new identities unseen in the training set. Moreover, the generalization capability of DeepID increases as more face classes are to be predicted at training. DeepID features are taken from the last hidden layer neuron activations of deep convolutional networks (ConvNets). When learned as classifiers to recognize about 10, 000 face identities in the training set and configured to keep reducing the neuron numbers along the feature extraction hierarchy, these deep ConvNets gradually form compact identity-related features in the top layers with only a small number of hidden neurons. The proposed features are extracted from various face regions to form complementary and over-complete representations. Any state-of-the-art classifiers can be learned based on these high-level representations for face verification. 97:45% verification accuracy on LFW is achieved with only weakly aligned faces. <s> BIB006 </s> Deep Learning Algorithms with Applications to Video Analytics for A Smart City: A Survey <s> IV. FACE RECOGNITION <s> This paper presents a new joint feature learning (JFL) approach to automatically learn feature representation from raw pixels for face recognition. Unlike many existing face recognition systems, where conventional feature descriptors, such as local binary patterns and Gabor features, are used for face representation, we propose an unsupervised feature learning method to learn hierarchical feature representation. Since different face regions have different physical characteristics, we propose to use different feature dictionaries to represent them, and to learn multiple yet related feature projection matrices for these regions simultaneously. Hence position-specific discriminative information can be exploited for face representation. Having learned these feature projections for different face regions, we perform spatial pooling for face patches within each region to enhance the representative power of the learned features. Moreover, we stack our JFL model into a deep architecture to exploit hierarchical information for feature representation and further improve the recognition performance. Experimental results on five widely used face data sets show the effectiveness of our proposed approach. <s> BIB007
Face recognition consists of two main tasks: face verification and face identification. The former aims to determine whether the given two faces belong to the same person. The latter is intended to find the identities of the given faces from the known face set. Recently, many deep learning based algorithms have achieved very promising results in these two face recognition tasks. We review some of them as follows. Huang et al. BIB003 propose to learn hierarchical features for face verification by using convolutional deep belief networks. The main contributions of this work are as follows: i) a local convolutional restricted Boltzmann machine is developed to adapt to the global structure in an object class (e.g. face); ii) deep learning is applied to local binary pattern representation BIB001 rather than raw pixel values to capture more complex characteristics of hand-crafted features; iii) learning the network architecture parameters is evaluated to be necessary for enhancing the multi-layer networks. The convolutional restricted Boltzmann machine used in the proposed method is illustrated in Figure 9 . It is reported that using the learned representations can achieve comparable performance with stateof-the-art methods using hand-crafted features. Actually, the Taigman et al. BIB005 propose a 3D face model based face alignment algorithm and a face representation learned from a nine-layer deep neural network. An overview of the architecture of the deep network is illustrated in Figure 10 . The first three convolutional layers are used to extract low-level features (e.g. edges and textures). The next three layers are locally connected to learn a different set of filters for each location of a face image since different regions have different local statistics. The top two layers are fully connected to capture correlations between features captured in different parts of a face image. At last, the output of the last layer is fed to a K-way softmax which predicts class labels. The objective of training is to maximize the probability of correct class by minimizing the cross-entropy loss for each training sample. It is shown that using the learned representations can achieve the near-human performance on the Labeled Faces in the Wild benchmark (LFW). Sun et al. BIB006 propose to learn so-called Deep hidden IDentity features (DeepID) for face verification. Figure 11 illustrates the feature extraction process. First, the local lowlevel features of an input face patch are extracted and fed into a ConvNet BIB002 . Then, the feature dimension gradually decreases to 160 through several feed-forward layers, during which more global and high-level features are learned. Last, the identity class (among 10, 000 classes) of the face patch is predicted directly by using the 160-dimensional DeepID. Rather than training a binary classifier for each face class, Sun et al. simultaneously classify all ConvNets regarding 10, 000 face identities. The advantages of this manipulation are as follows: i) effective features are extracted for face recognition by using the super learning capacity of neural networks; ii) the hidden features among all identities are shared by adding Fig. 11 . Illustration of the feature extraction process proposed in Sun et al. BIB006 . a strong regularization to ConvNets. It is reported that using the learned DeepID can achieve the near-human performance on the LFW dataset although only weakly aligned faces are used. Lu et al. BIB007 develop a joint feature learning approach to automatically learn hierarchical representation from raw pixels for face recognition. Figure 12 illustrates the basic idea of the proposed method. First, each face image is divided into several non-overlapping regions and feature weighting matrices are jointly learned. Then, the learned features in each region are pooled and represented as local histogram feature descriptors. Lastly, these local features are combined and concatenated into a longer feature vector for face representation. Moreover, the joint learning model is stacked into a deep architecture exploiting hierarchical information. As a result, the effectiveness of the proposed approach is demonstrated on five widely used face datasets. Face recognition has been widely used in security systems and human-machine interaction systems. It is still a challenge for computer to automatically identify or verify a person due to large variations, e.g. illumination, pose and expression. Deep learning can utilize big data for training deep architecture models so as to obtain more powerful features for representing faces. In future, face recognition systems in smart cities will largely rely on hierarchical features learned from deep models. Fig. 13 . Illustration of the architecture of the deep CNN proposed by Krizhevsky et al. BIB004 . It consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully connected layers with a 1000-way softmax.