Embedding methods have achieved success in face recognition by comparing facial features in a latent semantic space. However, in a fully unconstrained face setting, the features learned by the embedding model could be ambiguous or may not even be present in the input face, leading to noisy representations. We propose Probabilistic Face Embeddings (PFEs), which represent each face image as a Gaussian distribution in the latent space. The mean of the distribution estimates the most likely feature values while the variance shows the uncertainty in the feature values. Probabilistic solutions can then be naturally derived for matching and fusing PFEs using the uncertainty information. Empirical evaluation on different baseline models, training datasets and benchmarks show that the proposed method can improve the face recognition performance of deterministic embeddings by converting them into PFEs. The uncertainties estimated by PFEs also serve as good indicators of the potential matching accuracy, which are important for a risk-controlled recognition system.
Modeling faces as probabilistic distributions is not a new idea. In the field of face template video matching, there exists abundant literature on modeling the faces as probabilistic distributions @cite_32 @cite_5 , subspace @cite_1 or manifolds @cite_5 @cite_17 in the feature space. However, the input for such methods is a set of face images rather than a single face image, and they use a between-distribution similarity or distance measure, KL-divergence, for comparison, which does not penalize the uncertainty. Meanwhile, some studies @cite_43 @cite_15 have attempted to build a fuzzy model of a given face using the features of face parts. In comparison, the proposed PFE represents each single face image as a distribution in the latent space encoded by DNNs and we use an uncertainty-aware log likelihood score to compare the distributions.
@cite_1 @cite_32 @cite_43 @cite_5 @cite_15 @cite_17
1996939238 1506778248 2097729189 2122691893 2120007697 566612420
We introduce a novel method for face recognition from image sets. In our setting each test and training example is a set of images of an individual's face, not just a single image, so recognition decisions need to be based on comparisons of image sets. Methods for this have two main aspects: the models used to represent the individual image sets; and the similarity metric used to compare the models. Here, we represent images as points in a linear or affine feature space and characterize each image set by a convex geometric region (the affine or convex hull) spanned by its feature points. Set dissimilarity is measured by geometric distances (distances of closest approach) between convex models. To reduce the influence of outliers we use robust methods to discard input points that are far from the fitted model. The kernel trick allows the approach to be extended to implicit feature mappings, thus handling complex and nonlinear manifolds of face images. Experiments on two public face datasets show that our proposed methods outperform a number of existing state-of-the-art ones. We address the problem of face recognition from a large set of images obtained over time - a task arising in many surveillance and authentication applications. A set or a sequence of images provides information about the variability in the appearance of the face which can be used for more robust recognition. We discuss different approaches to the use of this information, and show that when cast as a statistical hypothesis testing problem, the classification task leads naturally to an information-theoretic algorithm that classifies sets of images using the relative entropy (Kullback-Leibler divergence) between the estimated density of the input set and that of stored collections of images for each class. We demonstrate the performance of the proposed algorithm on two medium-sized data sets of approximately frontal face images, and describe an application of the method as part of a view-independent recognition system. Pose variation remains to be a major challenge for real-world face recognition. We approach this problem through a probabilistic elastic matching method. We take a part based representation by extracting local features (e.g., LBP or SIFT) from densely sampled multi-scale image patches. By augmenting each feature with its location, a Gaussian mixture model (GMM) is trained to capture the spatial-appearance distribution of all face images in the training corpus. Each mixture component of the GMM is confined to be a spherical Gaussian to balance the influence of the appearance and the location terms. Each Gaussian component builds correspondence of a pair of features to be matched between two faces face tracks. For face verification, we train an SVM on the vector concatenating the difference vectors of all the feature pairs to decide if a pair of faces face tracks is matched or not. We further propose a joint Bayesian adaptation algorithm to adapt the universally trained GMM to better model the pose variations between the target pair of faces face tracks, which consistently improves face verification accuracy. Our experiments show that our method outperforms the state-of-the-art in the most restricted protocol on Labeled Face in the Wild (LFW) and the YouTube video face database by a significant margin. In many automatic face recognition applications, a set of a person's face images is available rather than a single image. In this paper, we describe a novel method for face recognition using image sets. We propose a flexible, semi-parametric model for learning probability densities confined to highly non-linear but intrinsically low-dimensional manifolds. The model leads to a statistical formulation of the recognition problem in terms of minimizing the divergence between densities estimated on these manifolds. The proposed method is evaluated on a large data set, acquired in realistic imaging conditions with severe illumination variation. Our algorithm is shown to match the best and outperform other state-of-the-art algorithms in the literature, achieving 94 recognition rate on average. Face is one of the important biometric identifier used for human recognition. The face recognition involves the computation of similarity between face images belonging to the determination of the identity of the face. The accurate recognition of face images is essential for the applications including credit card authentication, passport identification, internet security, criminal databases, biometric cryptosystems etc. Due to the increasing need for the surveillance and security related applications in access control, law enforcement, and information safety due to criminal activities, the research interest in the face recognition has grown considerably in the domain of the pattern recognition and image analysis. A number of approaches for face recognition have been proposed in the literature ( 2000), ( 1995). Many researchers have addressed face recognition based on geometrical features and template matching (Brunelli and Poggio, 1993). There are several well known face recognition methods such as Eigenfaces (Turk and Pentland 1991), Fisherfaces ( 1997), (Kim and Kitter 2005), Laplacianfaces ( 2005). The wavelet based Gabor function provide a favorable trade off between spatial resolution and frequency resolution (Gabor 1946). Gabor wavelets render superior representation for face recognition (Zhang, et al 2005), (Shan, et al 2004), (Olugbenga and Yang 2002). In recent survey, various potential problems and challenges in the face detection are explored (Yang, M.H., et al, 2002). Recent face detection methods based on data-driven learning techniques, such as the statistical modeling methods (Moghaddam and Pentland 1997), (Schneiderman, and Kanade, 2000), (Shih and Liu 2004), the statistical learning theory and SVM based methods (, 2001). Schneiderman and Kanade have developed the first algorithm that can reliably detect human faces with out-of-plane rotation and the first algorithm that can reliably detect passenger cars over a wide range of viewpoints (Schneiderman and Kanade 2000). The segmentation of potential face region in a digital image is a prelude to the face detection, since the search for the facial features is confined to the segmented face region. Several approaches have been used so far for the detection of face regions using skin color information. In (Wu, H.Q., et al, 1999), a face is detected using a fuzzy pattern matching method based on skin and hair color. This method has high detection rate, but it fails if the hair is not black and the face region is not elliptic. A face detection algorithm for color images using a skin-tone color model and facial features is The manifold of Symmetric Positive Definite (SPD) matrices has been successfully used for data representation in image set classification. By endowing the SPD manifold with Log-Euclidean Metric, existing methods typically work on vector-forms of SPD matrix logarithms. This however not only inevitably distorts the geometrical structure of the space of SPD matrix logarithms but also brings low efficiency especially when the dimensionality of SPD matrix is high. To overcome this limitation, we propose a novel metric learning approach to work directly on logarithms of SPD matrices. Specifically, our method aims to learn a tangent map that can directly transform the matrix logarithms from the original tangent space to a new tangent space of more discriminability. Under the tangent map framework, the novel metric learning can then be formulated as an optimization problem of seeking a Mahalanobis-like matrix, which can take the advantage of traditional metric learning techniques. Extensive evaluations on several image set classification tasks demonstrate the effectiveness of our proposed metric learning method.