Identifying kinship relations has garnered interest due to several applications such as organizing and tagging the enormous amount of videos being uploaded on the Internet. Existing research in kinship verification primarily focuses on kinship prediction with image pairs. In this research, we propose a new deep learning framework for kinship verification in unconstrained videos using a novel Supervised Mixed Norm AutoEncoder (SMNAE). This new autoencoder formulation introduces class-specific sparsity in the weight matrix. The proposed three-stage SMNAE based kinship verification framework utilizes the learned spatio-temporal representation in the video frames for verifying kinship in a pair of videos. A new kinship video (KIVI) database of more than 500 individuals with variations due to illumination, pose, occlusion, ethnicity, and expression is collected for this research. It comprises a total of 355 true kin video pairs with over 250 000 still frames. The effectiveness of the proposed framework is demonstrated on the KIVI database and six existing kinship databases. On the KIVI database, SMNAE yields video-based kinship verification accuracy of 83.18 which is at least 3.2 better than existing algorithms. The algorithm is also evaluated on six publicly available kinship databases and compared with best reported results. It is observed that the proposed SMNAE consistently yields best results on all the databases.
Table and Fig. showcase the research progression in the area of kinship verification. The problem of kinship verification using facial images was first proposed by @cite_43 . Since its origin, different frameworks and techniques have been applied for solving this problem which can be classified into two categories: approaches using handcrafted features and approaches based on learning features. Various handcrafted features have been incorporated to model kinship features and similarities in faces. @cite_43 employed low-level features such as eye color, skin color, and eye-to-nose distance for kinship verification. @cite_30 proposed a new spatial pyramid learning-based descriptor to encode kinship facial features. @cite_4 introduced Gabor-based gradient orientation pyramid representation method to learn kinship information in uncontrolled conditions. @cite_28 computed self-similarity descriptor to propose self-similarity representation of Weber face algorithm to classify a given pair of images as kin or non-kin. @cite_13 computed inheritable Fisher vector feature by applying the inheritable transformation on the Fisher vector for each image and performed kinship verification. @cite_16 demonstrated the efficacy of joint color-texture features from the luminance and chrominance channels of color images in kinship verification.
@cite_30 @cite_4 @cite_28 @cite_43 @cite_16 @cite_13
2108909793 2112999064 1992378626 1998230826 2578463952 2215847195
In this paper, we present an automatic kinship verification system based on facial image analysis under uncontrolled conditions. While a large number of studies on human face analysis have been performed in the literature, there are a few attempts on automatic face analysis for kinship verification, possibly due to lacking of such publicly available databases and great challenges of this problem. To this end, we collect a kinship face database by searching 400+ pairs of public figures and celebrities from the internet, and automatically detect them with the Viola-Jones face detector. Then, we propose a new spatial pyramid learning-based (SPLE) feature descriptor for face representation and apply support vector machine (SVM) for kinship verification. The proposed system has the following three characteristics: 1) no manual human annotation of face landmarks is required and the kinship information is automatically obtained from the original pair of images; 2) both local appearance information and global spatial information have been effectively utilized in the proposed SPLE feature descriptor, and better performance can be obtained than state-of-the-art feature descriptors in our application; 3) the performance of our proposed system is comparable to that of human observers. This paper presents a Gabor-based Gradient Orientation Pyramid (GGOP) feature representation method for kinship verification from facial images. First, we perform Gabor wavelet on each face image to obtain a set of Gabor magnitude (GM) feature images from different scales and orientations. Then, we extract the Gradient Orientation Pyramid (GOP) feature of each GM feature image and perform multiple feature fusion for kinship verification. When combined with the discriminative support vector machine (SVM) classifier, GGOP demonstrates the best performance in our experiments, in comparison with several state-of-the-art face feature descriptors. Experimental results are presented to show the efficacy of our proposed approach. Moreover, the performance of our proposed method is also comparable to that of human observers. Establishing kinship using images can be utilized as context information in different applications including face recognition. However, the process of automatically detecting kinship in facial images is a challenging and relatively less explored task. The reason for this includes limited availability of datasets as well as the inherent variations amongst kins. This paper presents a kinship classification algorithm that uses the local description of the pre-processed Weber face image. A kinship database is also prepared that contains images pertaining to 272 kin pairs. The database includes images of celebrities (and their kins) and has four ethnicity groups and seven kinship groups. The proposed algorithm outperforms an existing algorithm and yields a classification accuracy of 75.2 . We tackle the challenge of kinship verification using novel feature extraction and selection methods, automatically classifying pairs of face images as “related” or “unrelated” (in terms of kinship). First, we conducted a controlled online search to collect frontal face images of 150 pairs of public figures and celebrities, along with images of their parents or children. Next, we propose and evaluate a set of low-level image features for this classification problem. After selecting the most discriminative inherited facial features, we demonstrate a classification accuracy of 70.67 on a test set of image pairs using K-Nearest-Neighbors. Finally, we present an evaluation of human performance on this problem. Automatic kinship verification from faces aims to determine whether two persons have a biological kin relation or not by comparing their facial attributes. This is a challenging research problem that has recently received lots of attention from the research community. However, most of the proposed methods have mainly focused on analyzing only the luminance (i.e. gray-scale) of the face images, hence discarding the chrominance (i.e. color) information which can be a useful additional cue for verifying kin relationships. This paper investigates for the first time the usefulness of color information in the verification of kinship relationships from facial images. For this purpose, we extract joint color-texture features to encode both the luminance and the chrominance information in the color images. The kinship verification performance using joint color-texture analysis is then compared against counterpart approaches using only gray-scale information. Extensive experiments using different color spaces and texture features are conducted on two benchmark databases. Our results indicate that classifying color images consistently shows superior performance in three different color spaces. An innovative inheritable Fisher vector feature (IFVF) method is presented in this paper for kinship verification. Specifically, Fisher vector is first derived for each image by aggregating the densely sampled SIFT features in the opponent color space. Second, a new inheritable transformation, which maximizes the similarity between kinship images while minimizes that between non-kinship images for each image pair simultaneously, is learned based on the Fisher vectors. As a result, the IFVF is derived by applying the inheritable transformation on the Fisher vector for each image. Finally, a novel fractional power cosine similarity measure, which shows its theoretical roots in the Bayes decision rule for minimum error, is proposed for kinship verification. Experimental results on two representative kinship data sets, namely the KinFaceW-I and the KinFaceW-II data sets, show the feasibility of the proposed method.