Detecting Faces in Images : A Survey <s> Multiple Features <s> A basic concept of Multispectral Scanner data processing was developed for use in agricultural inventories; namely, to introduce spatial coordinates of each pixel into the vector description of the pixel and to use this information along with the spectral channel values in a conventional unsupervised clustering of the scene. The result is to isolate spectrally homogeneous field-like patches (called blobs). The spectral mean vector of a blob can be regarded as a defined feature and used in a conventional pattern recognition procedure. The benefits of use are: ease in locating training units in imagery; data compression of from 10 to 30 depending on the application; reduction of scanner noise and consequently potential improvements in classification/proportion estimation performances. <s> BIB001 </s> Detecting Faces in Images : A Survey <s> Multiple Features <s> The Hausdorff distance measures the extent to which each point of a model set lies near some point of an image set and vice versa. Thus, this distance can be used to determine the degree of resemblance between two objects that are superimposed on one another. Efficient algorithms for computing the Hausdorff distance between all possible relative positions of a binary image and a model are presented. The focus is primarily on the case in which the model is only allowed to translate with respect to the image. The techniques are extended to rigid motion. The Hausdorff distance computation differs from many other shape comparison methods in that no correspondence between the model and the image is derived. The method is quite tolerant of small position errors such as those that occur with edge detectors and other feature extraction methods. It is shown that the method extends naturally to the problem of comparing a portion of a model against an image. > <s> BIB002 </s> Detecting Faces in Images : A Survey <s> Multiple Features <s> This paper presents an automatic processing of human face from color images. The system works hierarchically from detecting the position of human face and its features (such as eyes, nose, mouth, etc.) to contours and feature points extraction. The position of human face and its parts are detected from the image by applying the integral projection method, which synthesize the color information (skin and hair color) and the edge information (intensity and sign). In order to extract the contour-line of face features we used a multiple active contour model with color information based energy terms. Facial feature points are decided based on the optimized contours. The proposed system is confirmed to be very effective and robust to deal with the image of faces in the complex background. <s> BIB003 </s> Detecting Faces in Images : A Survey <s> Multiple Features <s> Recognition of human faces out of still images or image sequences is a research field of fast increasing interest. At first, facial regions and facial features like eyes and mouth have to be extracted. In the present paper we propose an approach that copes with problems of these first two steps. We perform face localization based on the observation that human faces are characterized by their oval shape and skin-color, also in the case of varying light conditions. For that we segment faces by evaluating shape and color (HSV) information. Then face hypotheses are verified by searching for facial features inside of the face-like regions. This is done by applying morphological operations and minima localization to intensity images. <s> BIB004 </s> Detecting Faces in Images : A Survey <s> Multiple Features <s> Describes a transform to extract image regions at all geometric and photometric scales. It is argued that linear approaches have the shortcoming that they require a priori models of region shape. The proposed transform avoids this by letting the structure emerge, bottom-up, from interactions among pixels. The transform involves global computations on pairs of pixels followed by vector integration of the results. An attraction force field is computed over the image in which pixels belonging to the same region are mutually attracted and the region is characterized by a convergent flow. It is shown that the transform possesses properties that allow multiscale segmentation, or extraction of original, unblurred structure at all different geometric and photometric scales present. This is in contrast with much previous work wherein multiscale structure is viewed as the smoothed structure in a multiscale signal decimation. Scale is an integral parameter of the force computation, and the number and values of scale parameters associated with the image can be estimated automatically. Regions are detected at all a priori unknown scales resulting in automatic construction of a segmentation tree, in which each pixel is annotated with descriptions of all the regions it belongs to. Transform properties are presented for piecewise-constant images but hold for more general ones. Thus the proposed method is intended as a solution to the problem of multiscale, integrated edge and region detection, or low-level image segmentation. Experimental results are given. <s> BIB005 </s> Detecting Faces in Images : A Survey <s> Multiple Features <s> This paper proposes a scale invariant face detection method which combines higher-order local autocorrelation (HLAC) features extracted from a log-polar transformed image with linear discriminant analysis for "face" and "not face" classification. Since HLAC features of log-polar images are sensitive to shifts of a face, we utilize this property and develop a face detection method. HLAC features extracted from a log-polar image become scale and rotation invariant because scalings and rotations of a face are expressed as shifts in a log-polar image (coordinate). By combining these features with the linear discriminant analysis which is extended to treat "face" and "not face" classes, a scale invariant face detection system can be realized. <s> BIB006 </s> Detecting Faces in Images : A Survey <s> Multiple Features <s> We use a skin color model based on the Muhulanobis metric and a shape analysis based on invariant moments to automatically detect and locate human faces in two-dimensional natural scene images. First, color segmentation of an input image is performed by thresholding in a perceptually plausible hue-saturation color space where the effects of the variability of human skin color and the dependency of chrominance on changes in illumination are reduced. We then group regions of the resulting binary image which have been classified as candidates into clusters of connected pixels. Performing median filtering on the image and discarding the smallest remaining clusters ensures that only a small number of clusters will be used for further analysis. Fully translation-, scale- anti in-plane rotation invariant moments are calculated for each remaining cluster. Finally, in order to distinguish faces from distractors, a multilayer perceptron neural network is used with the invariant moments as the input vector. Supervised learning of the network is implemented with the backpropagation algorithm, at first for frontal views of faces. Preliminary results show the efficiency of the combination of color segmentation and of invariant moments in detecting faces with a large variety of poses and against relatively complex backgrounds. <s> BIB007 </s> Detecting Faces in Images : A Survey <s> Multiple Features <s> We use a skin color model based on the Mahalanobis metric and a shape analysis based on invariant Fourier-Mellin moments to automatically detect and locate human faces in two-dimensional complex scene images. First, color segmentation of an input image is performed by thresholding in a normalized hue-saturation. Color space where the effects of the variability of human skin color and the dependency of chrominance on changes in illumination are reduced. We then group regions of the resulting binary image that have been classified as face candidates into clusters of connected pixels. Discarding the smallest clusters in the image ensures that only a small number of clusters will be used for further analysis. Fully translation-, scale- and in-plane rotation-invariant moments are calculated for each remaining cluster Finally, in order to distinguish faces from distractors, a multilayer perceptron neural network is used with the invariant moments as the input vector supervised learning of the network is implemented with the backpropagation algorithm, at first for frontal views of faces. Preliminary results show the efficiency of the combination of color segmentation and of invariant moments in detecting faces with a large variety of poses and against relatively complex backgrounds. <s> BIB008 </s> Detecting Faces in Images : A Survey <s> Multiple Features <s> We describe an algorithm for detecting human faces and facial features, such as the location of the eyes, nose and mouth. First, a supervised pixel-based color classifier is employed to mark all pixels that are within a prespecified distance of ''skin color'', which is computed from a training set of skin patches. This color-classification map is then smoothed by Gibbs random field model-based filters to define skin regions. An ellipse model is fit to each disjoint skin region. Finally, we introduce symmetry-based cost functions to search the center of the eyes, tip of nose, and center of mouth within ellipses whose aspect ratio is similar to that of a face. <s> BIB009 </s> Detecting Faces in Images : A Survey <s> Multiple Features <s> As we know, a robust approach to face and facial features detection must be able to handle the variation issues such as changes in imaging conditions, face appearances and image contents. Here we present a method which utilizes color, local symmetry and geometry information of human face based on various models. The algorithm first detects most likely face regions or ROIs (Region-Of-Interest) from the image using face color model and face outline model, produces a face color similarity map. Then it performs local symmetry detection within these ROIs to obtain a local symmetry similarity map. These two maps are fused to obtain potential facial feature points. Finally similarity matching is performed to identify faces between the fusion map and face geometry model under affine transformation. The output results are the detected faces with confidence values. Experimental results have demonstrated its validity and robustness to identify faces under certain variations. <s> BIB010 </s> Detecting Faces in Images : A Survey <s> Multiple Features <s> We propose a new method to detect human faces in color images. A human skin color model is built to capture the chromatic properties based on multivariate statistical analysis. Given a color image, multiscale segmentation is used to generate homogeneous regions at multiple different scales. From the coarsest to the finest scale, regions of skin color are merged until the shape is approximately elliptic. Postprocessing is performed to determine whether a merged region contains a human face and include the facial features of non-skin color such as eyes and mouth if necessary. Experimental results show that human faces in color images can be detected regardless of size, orientation and viewpoint. <s> BIB011 </s> Detecting Faces in Images : A Survey <s> Multiple Features <s> This paper proposes an object oriented face detection method using range and color information. Objects are segmented from the background using the stereo disparity histogram that represents the range information of the objects. A matching pixel count (MPC) disparity measure is introduced to enhance the matching accuracy, and removes the effect of unexpected noise in the boundary region. For the high-performance implementation of the MPC disparity histogram, redundancy operations inherent to the area-based search operation are removed. To detect facial regions among segmented objects, a skin-color transform technique is used with the generalized face color distribution (GFCD) modeled by a 2D Gaussian function in a normalized color space. Using GFCD, the input color image can be transformed into a gray-level image enhancing only facial color components. To detect facial information only in the defined range, both results from range segmentation and color transforms are combined effectively. The experimental results show that the proposed algorithm works well in various environments with multiple human objects. Moreover, the processing time for a test image is not exceeding 2 seconds in general purpose workstations. The range information of the objects can be useful in MPEG-4 where natural and synthetic images can be mixed and synthesized. <s> BIB012 </s> Detecting Faces in Images : A Survey <s> Multiple Features <s> We present an approach to real-time person tracking in crowded and/or unknown environments using integration of multiple visual modalities. We combine stereo, color, and face detection modules into a single robust system, and show an initial application in an interactive, face-responsive display. Dense, real-time stereo processing is used to isolate users from other objects and people in the background. Skin-hue classification identifies and tracks likely body parts within the silhouette of a user. Face pattern detection discriminates and localizes the face within the identified body parts. Faces and bodies of users are tracked over several temporal scales: short-term (user stays within the field of view), medium-term (user exits/reenters within minutes), and long term (user returns after hours or days). Short-term tracking is performed using simple region position and size correspondences, while medium and long-term tracking are based on statistics of user appearance. We discuss the failure modes of each individual module, describe our integration method, and report results with the complete system in trials with thousands of users. <s> BIB013
Recently, numerous methods that combine several facial features have been proposed to locate or detect faces. Most of them utilize global features such as skin color, size, and shape to find face candidates, and then verify these candidates using local, detailed features such as eye brows, nose, and hair. A typical approach begins with the detection of skin-like regions as described in Section 2.2.3. Next, skin-like pixels are grouped together using connected component analysis or clustering algorithms. If the shape of a connected region has an elliptic or oval shape, it becomes a face candidate. Finally, local features are used for verification. However, others, such as  , BIB006 , have used different sets of features. Yachida et al. presented a method to detect faces in color images using fuzzy theory  , BIB003 ,  . They used two fuzzy models to describe the distribution of skin and hair color in CIE XYZ color space. Five (one frontal and four side views) head-shape models are used to abstract the appearance of faces in images. Each shape model is a 2D pattern consisting of m Â n square cells where each cell may contain several pixels. Two properties are assigned to each cell: the skin proportion and the hair proportion, which indicate the ratios of the skin area (or the hair area) within the cell to the area of the cell. In a test image, each pixel is classified as hair, face, hair/face, and hair/background based on the distribution models, thereby generating skin-like and hair-like regions. The head shape models are then compared with the extracted skin-like and hair-like regions in a test image. If they are similar, the detected region becomes a face candidate. For verification, eye-eyebrow and nose-mouth features are extracted from a face candidate using horizontal edges. Sobottka and Pitas proposed a method for face localization and facial feature extraction using shape and color BIB004 . First, color segmentation in HSV space is performed to locate skin-like regions. Connected components are then determined by region growing at a coarse resolution. For each connected component, the best fit ellipse is computed using geometric moments. Connected components that are well approximated by an ellipse are selected as face candidates. Subsequently, these candidates are verified by searching for facial features inside of the connected components. Features, such as eyes and mouths, are extracted based on the observation that they are darker than the rest of a face. In BIB007 , BIB008 , a Gaussian skin color model is used to classify skin color pixels. To characterize the shape of the clusters in the binary image, a set of 11 lowest-order geometric moments is computed using Fourier and radial Mellin transforms. For detection, a neural network is trained with the extracted geometric moments. Their experiments show a detection rate of 85 percent based on a test set of 100 images. The symmetry of face patterns has also been applied to face localization BIB009 . Skin/nonskin classification is carried out using the class-conditional density function in YES color space followed by smoothing in order to yield contiguous regions. Next, an elliptical face template is used to determine the similarity of the skin color regions based on Hausdorff distance BIB002 . Finally, the eye centers are localized using several cost functions which are designed to take advantage of the inherent symmetries associated with face and eye locations. The tip of the nose and the center of the mouth are then located by utilizing the distance between the eye centers. One drawback is that it is effective only for a single frontal-view face and when both eyes are visible. A similar method using color and local symmetry was presented in BIB010 . In contrast to pixel-based methods, a detection method based on structure, color, and geometry was proposed in BIB011 . First, multiscale segmentation BIB005 is performed to extract homogeneous regions in an image. Using a Gaussian skin color model, regions of skin tone are extracted and grouped into ellipses. A face is detected if facial features such as eyes and mouth exist within these elliptic regions. Experimental results show that this method is able to detect faces at different orientations with facial features such as beard and glasses. Kauth et al. proposed a blob representation to extract a compact, structurally meaningful description of multispectral satellite imagery BIB001 . A feature vector at each pixel is formed by concatenating the pixel's image coordinates to the pixel's spectral (or textural) components; pixels are then clustered using this feature vector to form coherent connected regions, or "blobs." To detect faces, each feature vector consists of the image coordinates and normalized chrominance, i.e., X ¼ ðx; y; r rþgþb ; g rþgþb Þ  ,  . A connectivity algorithm is then used to grow blobs and the resulting skin blob whose size and shape is closest to that of a canonical face is considered as a face. Range and color have also been employed for face detection by Kim et al. BIB012 . Disparity maps are computed and objects are segmented from the background with a disparity histogram using the assumption that background pixels have the same depth and they outnumber the pixels in the foreground objects. Using a Gaussian distribution in normalized RGB color space, segmented regions with a skinlike color are classified as faces. A similar approach has been proposed by Darrell et al. for face detection and tracking BIB013 .