Identifying kinship relations has garnered interest due to several applications such as organizing and tagging the enormous amount of videos being uploaded on the Internet. Existing research in kinship verification primarily focuses on kinship prediction with image pairs. In this research, we propose a new deep learning framework for kinship verification in unconstrained videos using a novel Supervised Mixed Norm AutoEncoder (SMNAE). This new autoencoder formulation introduces class-specific sparsity in the weight matrix. The proposed three-stage SMNAE based kinship verification framework utilizes the learned spatio-temporal representation in the video frames for verifying kinship in a pair of videos. A new kinship video (KIVI) database of more than 500 individuals with variations due to illumination, pose, occlusion, ethnicity, and expression is collected for this research. It comprises a total of 355 true kin video pairs with over 250 000 still frames. The effectiveness of the proposed framework is demonstrated on the KIVI database and six existing kinship databases. On the KIVI database, SMNAE yields video-based kinship verification accuracy of 83.18 which is at least 3.2 better than existing algorithms. The algorithm is also evaluated on six publicly available kinship databases and compared with best reported results. It is observed that the proposed SMNAE consistently yields best results on all the databases.
* 1.1 In 2013, @cite_6 introduced the problem of kinship verification in videos. They computed spatio-temporal facial-dynamics features of high quality videos to model kinship characteristics between two individuals. They used UvA-NEMO Smile database for experimental purposes where smile based dynamic spatio-temporal features and complete local binary patterns on three orthogonal planes @cite_29 were extracted and used for kinship verification. This database was initially developed for expression variations and was captured in controlled settings. Recently, Dibeklioglu @cite_21 introduced visual transformation between the facial appearance of kin for the task of video-based kinship verification.
@cite_29 @cite_21 @cite_6
1998022503 2776258980 2164322079
In this paper we propose the first method known to the authors that successfully differentiates spontaneous from posed facial expressions using a realistic training corpus. We propose a new spatiotemporal local texture descriptor (CLBP-TOP) that outperforms other descriptors. We demonstrate that our temporal interpolation and visual near-infrared fusion methods improve the differentiation performance. Finally, we propose a new generic facial expression recognition framework that subdivides the facial expression recognition problem into a cascade of smaller tasks that are simpler to tackle. The system is the first to differentiate spontaneous from posed facial expressions with a realistic corpus and achieves promising results. Automatic kinship verification from facial information is a relatively new and open research problem in computer vision. This paper explores the possibility of learning an efficient facial representation for video-based kinship verification by exploiting the visual transformation between facial appearance of kin pairs. To this end, a Siamese-like coupled convolutional encoder-decoder network is proposed. To reveal resemblance patterns of kinship while discarding the similarity patterns that can also be observed between people who do not have a kin relationship, a novel contrastive loss function is defined in the visual appearance space. For further optimization, the learned representation is fine-tuned using a feature-based contrastive loss. An expression matching procedure is employed in the model to minimize the negative influence of expression differences between kin pairs. Each kin video is analyzed by a sliding temporal window to leverage short-term facial dynamics. The effectiveness of the proposed method is assessed on seven different kin relationships using smile videos of kin pairs. On the average, 93:65 verification accuracy is achieved, improving the state of the art. Kinship verification from facial appearance is a difficult problem. This paper explores the possibility of employing facial expression dynamics in this problem. By using features that describe facial dynamics and spatio-temporal appearance over smile expressions, we show that it is possible to improve the state of the art in this problem, and verify that it is indeed possible to recognize kinship by resemblance of facial expressions. The proposed method is tested on different kin relationships. On the average, 72.89 verification accuracy is achieved on spontaneous smiles.