Identifying kinship relations has garnered interest due to several applications such as organizing and tagging the enormous amount of videos being uploaded on the Internet. Existing research in kinship verification primarily focuses on kinship prediction with image pairs. In this research, we propose a new deep learning framework for kinship verification in unconstrained videos using a novel Supervised Mixed Norm AutoEncoder (SMNAE). This new autoencoder formulation introduces class-specific sparsity in the weight matrix. The proposed three-stage SMNAE based kinship verification framework utilizes the learned spatio-temporal representation in the video frames for verifying kinship in a pair of videos. A new kinship video (KIVI) database of more than 500 individuals with variations due to illumination, pose, occlusion, ethnicity, and expression is collected for this research. It comprises a total of 355 true kin video pairs with over 250 000 still frames. The effectiveness of the proposed framework is demonstrated on the KIVI database and six existing kinship databases. On the KIVI database, SMNAE yields video-based kinship verification accuracy of 83.18 which is at least 3.2 better than existing algorithms. The algorithm is also evaluated on six publicly available kinship databases and compared with best reported results. It is observed that the proposed SMNAE consistently yields best results on all the databases.
* 1.1 In 2013, @cite_6 introduced the problem of kinship verification in videos. They computed spatio-temporal facial-dynamics features of high quality videos to model kinship characteristics between two individuals. They used UvA-NEMO Smile database for experimental purposes where smile based dynamic spatio-temporal features and complete local binary patterns on three orthogonal planes @cite_29 were extracted and used for kinship verification. This database was initially developed for expression variations and was captured in controlled settings. Recently, Dibeklioglu @cite_21 introduced visual transformation between the facial appearance of kin for the task of video-based kinship verification.
@cite_29 @cite_21 @cite_6
2776258980 2164322079 2520903215 2108909793
Automatic kinship verification from facial information is a relatively new and open research problem in computer vision. This paper explores the possibility of learning an efficient facial representation for video-based kinship verification by exploiting the visual transformation between facial appearance of kin pairs. To this end, a Siamese-like coupled convolutional encoder-decoder network is proposed. To reveal resemblance patterns of kinship while discarding the similarity patterns that can also be observed between people who do not have a kin relationship, a novel contrastive loss function is defined in the visual appearance space. For further optimization, the learned representation is fine-tuned using a feature-based contrastive loss. An expression matching procedure is employed in the model to minimize the negative influence of expression differences between kin pairs. Each kin video is analyzed by a sliding temporal window to leverage short-term facial dynamics. The effectiveness of the proposed method is assessed on seven different kin relationships using smile videos of kin pairs. On the average, 93:65 verification accuracy is achieved, improving the state of the art. Kinship verification from facial appearance is a difficult problem. This paper explores the possibility of employing facial expression dynamics in this problem. By using features that describe facial dynamics and spatio-temporal appearance over smile expressions, we show that it is possible to improve the state of the art in this problem, and verify that it is indeed possible to recognize kinship by resemblance of facial expressions. The proposed method is tested on different kin relationships. On the average, 72.89 verification accuracy is achieved on spontaneous smiles. Kinship verification has a number of applications such as organizing large collections of images and recognizing resemblances among humans. In this paper, first, a human study is conducted to understand the capabilities of human mind and to identify the discriminatory areas of a face that facilitate kinship-cues. The visual stimuli presented to the participants determine their ability to recognize kin relationship using the whole face as well as specific facial regions. The effect of participant gender and age and kin-relation pair of the stimulus is analyzed using quantitative measures such as accuracy, discriminability index @math , and perceptual information entropy. Utilizing the information obtained from the human study, a hierarchical kinship verification via representation learning (KVRL) framework is utilized to learn the representation of different face regions in an unsupervised manner. We propose a novel approach for feature representation termed as filtered contractive deep belief networks ( fc DBN). The proposed feature representation encodes relational information present in images using filters and contractive regularization penalty. A compact representation of facial images of kin is extracted as an output from the learned model and a multi-layer neural network is utilized to verify the kin accurately. A new WVU kinship database is created, which consists of multiple images per subject to facilitate kinship verification. The results show that the proposed deep learning framework (KVRL- fc DBN) yields the state-of-the-art kinship verification accuracy on the WVU kinship database and on four existing benchmark data sets. Furthermore, kinship information is used as a soft biometric modality to boost the performance of face verification via product of likelihood ratio and support vector machine based approaches. Using the proposed KVRL- fc DBN framework, an improvement of over 20 is observed in the performance of face verification. In this paper, we present an automatic kinship verification system based on facial image analysis under uncontrolled conditions. While a large number of studies on human face analysis have been performed in the literature, there are a few attempts on automatic face analysis for kinship verification, possibly due to lacking of such publicly available databases and great challenges of this problem. To this end, we collect a kinship face database by searching 400+ pairs of public figures and celebrities from the internet, and automatically detect them with the Viola-Jones face detector. Then, we propose a new spatial pyramid learning-based (SPLE) feature descriptor for face representation and apply support vector machine (SVM) for kinship verification. The proposed system has the following three characteristics: 1) no manual human annotation of face landmarks is required and the kinship information is automatically obtained from the original pair of images; 2) both local appearance information and global spatial information have been effectively utilized in the proposed SPLE feature descriptor, and better performance can be obtained than state-of-the-art feature descriptors in our application; 3) the performance of our proposed system is comparable to that of human observers.