Detecting Faces in Images : A Survey <s> Neural Networks <s> The authors present a time-delay neural network (TDNN) approach to phoneme recognition which is characterized by two important properties: (1) using a three-layer arrangement of simple computing units, a hierarchy can be constructed that allows for the formation of arbitrary nonlinear decision surfaces, which the TDNN learns automatically using error backpropagation; and (2) the time-delay arrangement enables the network to discover acoustic-phonetic features and the temporal relationships between them independently of position in time and therefore not blurred by temporal shifts in the input. As a recognition task, the speaker-dependent recognition of the phonemes B, D, and G in varying phonetic contexts was chosen. For comparison, several discrete hidden Markov models (HMM) were trained to perform the same task. Performance evaluation over 1946 testing tokens from three speakers showed that the TDNN achieves a recognition rate of 98.5% correct while the rate obtained by the best of the HMMs was only 93.7%. > <s> BIB001 </s> Detecting Faces in Images : A Survey <s> Neural Networks <s> Abstract A method for automatic detection and localization of faces on digital images is proposed. The method is based on learning by example and multi-resolution analysis of digital images. Special emphasis is put on the management of the learning data, in order to improve the performances. Various experimental results, obtained by using a Multi-Layer Perceptron (MLP) as a classifier, are provided. <s> BIB002 </s> Detecting Faces in Images : A Survey <s> Neural Networks <s> We present a neural network-based face detection system. A retinally connected neural network examines small windows of an image, and decides whether each window contains a face. The system arbitrates between multiple networks to improve performance over a single network. We use a bootstrap algorithm for training, which adds false detections into the training set as training progresses. This eliminates the difficult task of manually selecting non-face training examples, which must be chosen to span the entire space of non-face images. Comparisons with another state-of-the-art face detection system are presented; our system has better performance in terms of detection and false-positive rates. <s> BIB003 </s> Detecting Faces in Images : A Survey <s> Neural Networks <s> This paper summarizes a research effort in human face detection. A system to locate human faces in images, especially when used as a front-end for a human face identification system, could have many applications in the law enforcement and security professions. The approach presented here is a hybrid system using an edge-enhancing preprocessor and four back-propagation neural networks arranged in a hierarchical structure. The method proposed successfully detected faces wearing glasses and all faces in images which contained multiple faces. The results obtained are reported along with a discussion for improving the system. <s> BIB004 </s> Detecting Faces in Images : A Survey <s> Neural Networks <s> Both visual and acoustical informations provide effective means of telecommunication between persons. In this context, the face is the most important part of the person both visually and acoustically. We describe how the cooperation of image and audio processing allows to track a person's face and to collect the audio information it produces. We present detection techniques of regions of interest (e.g. Moving regions of skin color), coupled with a neural network based face detector with a low false alarm rate, to locate and track faces. The system is connected to a nine microphone array adaptive beam forming which performs immediate beam forming. Visual and acoustical informations from the speaker face are thus obtained in real time. <s> BIB005 </s> Detecting Faces in Images : A Survey <s> Neural Networks <s> A new learning model based on autoassociative neural networks is developped and applied to face detection. To extend the detection ability in orientation and to decrease the number of false alarms, different combinations of networks are tested: ensemble, conditional ensemble and conditional mixture of networks. The use of a conditional mixture of networks allows to obtain state of the art results on different benchmark face databases. <s> BIB006 </s> Detecting Faces in Images : A Survey <s> Neural Networks <s> This paper proposes a face recognition system, based on probabilistic decision-based neural networks (PDBNN). With technological advance on microelectronic and vision system, high performance automatic techniques on biometric recognition are now becoming economically feasible. Among all the biometric identification methods, face recognition has attracted much attention in recent years because it has potential to be most nonintrusive and user-friendly. The PDBNN face recognition system consists of three modules: First, a face detector finds the location of a human face in an image. Then an eye localizer determines the positions of both eyes in order to generate meaningful feature vectors. The facial region proposed contains eyebrows, eyes, and nose, but excluding mouth (eye-glasses will be allowed). Lastly, the third module is a face recognizer. The PDBNN can be effectively applied to all the three modules. It adopts a hierarchical network structures with nonlinear basis functions and a competitive credit-assignment scheme. The paper demonstrates a successful application of PDBNN to face recognition applications on two public (FERET and ORL) and one in-house (SCR) databases. Regarding the performance, experimental results on three different databases such as recognition accuracies as well as false rejection and false acceptance rates are elaborated. As to the processing speed, the whole recognition process (including PDBNN processing for eye localization, feature extraction, and classification) consumes approximately one second on Sparc10, without using hardware accelerator or co-processor. <s> BIB007 </s> Detecting Faces in Images : A Survey <s> Neural Networks <s> A generative neural network model, constrained by non-face examples chosen by an iterative algorithm, is applied to face detection. To extend the detection ability in orientation and to decrease the number of false alarms, different combinations of networks are tested: ensemble, conditional ensemble and conditional mixture of networks. The use of a conditional mixture of networks obtains better results on different benchmark face databases than state-of-the-art. <s> BIB008 </s> Detecting Faces in Images : A Survey <s> Neural Networks <s> A real-time system is described for automatic detection and tracking of multiple persons, in the context of video-conferencing systems. This system, called MULTRAK (multiperson locating and tracking automatic kernel) is able to continuously detect and track the position of faces in its field of view. The heart of the system as a modular neural network based face detector, giving fast and accurate face detection. <s> BIB009 </s> Detecting Faces in Images : A Survey <s> Neural Networks <s> We present an example-based learning approach for locating vertical frontal views of human faces in complex scenes. The technique models the distribution of human face patterns by means of a few view-based "face" and "nonface" model clusters. At each image location, a difference feature vector is computed between the local image pattern and the distribution-based model. A trained classifier determines, based on the difference feature vector measurements, whether or not a human face exists at the current image location. We show empirically that the distance metric we adopt for computing difference feature vectors, and the "nonface" clusters we include in our distribution-based model, are both critical for the success of our system. <s> BIB010 </s> Detecting Faces in Images : A Survey <s> Neural Networks <s> We present a neural network-based upright frontal face detection system. A retinally connected neural network examines small windows of an image and decides whether each window contains a face. The system arbitrates between multiple networks to improve performance over a single network. We present a straightforward procedure for aligning positive face examples for training. To collect negative examples, we use a bootstrap algorithm, which adds false detections into the training set as training progresses. This eliminates the difficult task of manually selecting nonface training examples, which must be chosen to span the entire space of nonface images. Simple heuristics, such as using the fact that faces rarely overlap in images, can further improve the accuracy. Comparisons with several other state-of-the-art face detection systems are presented, showing that our system has comparable performance in terms of detection and false-positive rates. <s> BIB011 </s> Detecting Faces in Images : A Survey <s> Neural Networks <s> In this paper, we present a neural network-based face detection system. Unlike similar systems which are limited to detecting upright, frontal faces, this system detects faces at any degree of rotation in the image plane. The system employs multiple networks; a "router" network first processes each input window to determine its orientation and then uses this information to prepare the window for one or more "detector" networks. We present the training methods for both types of networks. We also perform sensitivity analysis on the networks, and present empirical results on a large test set. Finally, we present preliminary results for detecting faces rotated out of the image plane, such as profiles and semi-profiles. <s> BIB012 </s> Detecting Faces in Images : A Survey <s> Neural Networks <s> Nonlinear principal component analysis is a novel technique for multivariate data analysis, similar to the well-known method of principal component analysis. NLPCA, like PCA, is used to identify and remove correlations among problem variables as an aid to dimensionality reduction, visualization, and exploratory data analysis. While PCA identifies only linear correlations between variables, NLPCA uncovers both linear and nonlinear correlations, without restriction on the character of the nonlinearities present in the data. NLPCA operates by training a feedforward neural network to perform the identity mapping, where the network inputs are reproduced at the output layer. The network contains an internal “bottleneck” layer (containing fewer nodes than input or output layers), which forces the network to develop a compact representation of the input data, and two additional hidden layers. The NLPCA method is demonstrated using time-dependent, simulated batch reaction data. Results show that NLPCA successfully reduces dimensionality and produces a feature space map resembling the actual distribution of the underlying system parameters. <s> BIB013
Neural networks have been applied successfully in many pattern recognition problems, such as optical character recognition, object recognition, and autonomous robot driving. Since face detection can be treated as a two class pattern recognition problem, various neural network architectures have been proposed. The advantage of using neural networks for face detection is the feasibility of training a system to capture the complex class conditional density of face patterns. However, one drawback is that the network architecture has to be extensively tuned (number of layers, number of nodes, learning rates, etc.) to get exceptional performance. An early method using hierarchical neural networks was proposed by Agui et al.  . The first stage consists of two parallel subnetworks in which the inputs are intensity values from an original image and intensity values from filtered image using a 3 Â 3 Sobel filter. The inputs to the second stage network consist of the outputs from the subnetworks and extracted feature values such as the standard deviation of the pixel values in the input pattern, a ratio of the number of white pixels to the total number of binarized pixels in a window, and geometric moments. An output value at the second stage indicates the presence of a face in the input region. Experimental results show that this method is able to detect faces if all faces in the test images have the same size. Propp and Samal developed one of the earliest neural networks for face detection  . Their network consists of four layers with 1,024 input units, 256 units in the first hidden layer, eight units in the second hidden layer, and two output units. A similar hierarchical neural network is later proposed by BIB004 . The early method by Soulie et al.  scans an input image with a time-delay neural network BIB001 (with a receptive field of 20 Â 25 pixels) to detect faces. To cope with size variation, the input image is decomposed using wavelet transforms. They reported a false negative rate of 2.7 percent and false positive rate of 0.5 percent from a test of 120 images. In  , Vaillant et al. used convolutional neural networks to detect faces in images. Examples of face and nonface images of 20 Â 20 pixels are first created. One neural network is trained to find approximate locations of faces at some scale. Another network is trained to determine the exact position of faces at some scale. Given an image, areas which may contain faces are selected as face candidates by the first network. These candidates are verified by the second network. Burel and Carel BIB002 proposed a neural network for face detection in which the large number of training examples of faces and nonfaces are compressed into fewer examples using a Kohonen's SOM algorithm  . A multilayer perceptron is used to learn these examples for face/background classification. The detection phase consists of scanning each image at various resolution. For each location and size of the scanning window, the contents are normalized to a standard size, and the intensity mean and variance are scaled to reduce the effects of lighting conditions. Each normalized window is then classified by an MLP. Feraud and Bernier presented a detection method using autoassociative neural networks BIB006 , BIB008 ,  . The idea is based on BIB013 which shows an autoassociative network with five layers is able to perform a nonlinear principal component analysis. One autoassociative network is used to detect frontal-view faces and another one is used to detect faces turned up to 60 degrees to the left and right of the frontal view. A gating network is also utilized to assign weights to frontal and turned face detectors in an ensemble of autoassociative networks. On a small test set of 42 images, they report a detection rate similar to BIB003 . The method has also been employed in LISTEN BIB005 and MULTRAK BIB009 . Lin et al. presented a face detection system using probabilistic decision-based neural network (PDBNN) BIB007 . The architecture of PDBNN is similar to a radial basis function (RBF) network with modified learning rules and probabilistic interpretation. Instead of converting a whole face image into a training vector of intensity values for the neural network, they first extract feature vectors based on intensity and edge information in the facial region that contains eyebrows, eyes, and nose. The extracted two feature vectors are fed into two PDBNN's and the fusion of the outputs determine the classification result. Based on a set of 23 images provided by Sung and Poggio BIB010 , their experimental results show comparable performance with the other leading neural network-based face detectors BIB010 , BIB011 . Among all the face detection methods that used neural networks, the most significant work is arguably done by Rowley et al. BIB011 , BIB003 , BIB011 . A multilayer neural network is used to learn the face and nonface patterns from face/ Fig. 9 . Prototype of each face class using Kohonen's SOM by Yang et al.  . Each prototype corresponds to the center of a cluster. nonface images (i.e., the intensities and spatial relationships of pixels) whereas Sung  used a neural network to find a discriminant function to classify face and nonface patterns using distance measures. They also used multiple neural networks and several arbitration methods to improve performance, while Burel and Carel BIB002 used a single network, and Vaillant et al.  used two networks for classification. There are two major components: multiple neural networks (to detect face patterns) and a decisionmaking module (to render the final decision from multiple detection results). As shown in Fig. 10 , the first component of this method is a neural network that receives a 20 Â 20 pixel region of an image and outputs a score ranging from -1 to 1. Given a test pattern, the output of the trained neural network indicates the evidence for a nonface (close to -1) or face pattern (close to 1). To detect faces anywhere in an image, the neural network is applied at all image locations. To detect faces larger than 20 Â 20 pixels, the input image is repeatedly subsampled, and the network is applied at each scale. Nearly 1,050 face samples of various sizes, orientations, positions, and intensities are used to train the network. In each training image, the eyes, tip of the nose, corners, and center of the mouth are labeled manually and used to normalize the face to the same scale, orientation, and position. The second component of this method is to merge overlapping detection and arbitrate between the outputs of multiple networks. Simple arbitration schemes such as logic operators (AND/OR) and voting are used to improve performance. Rowley et al. BIB011 reported several systems with different arbitration schemes that are less computationally expensive than Sung and Poggio's system and have higher detection rates based on a test set of 24 images containing 144 faces. One limitation of the methods by Rowley BIB011 and by Sung  is that they can only detect upright, frontal faces. Recently, Rowley et al. BIB012 extended this method to detect rotated faces using a router network which processes each input window to determine the possible face orientation and then rotates the window to a canonical orientation; the rotated window is presented to the neural networks as described above. However, the new system has a lower detection rate on upright faces than the upright detector. Nevertheless, the system is able to detect 76.9 percent of faces over two large test sets with a small number of false positives.