Recently, there has been emerging research on the interpretation
of face recognition systems[1]}[2]}[3]}[4]}[5]}[6]}[7]}. Visual psychophysics [5]} have provided a set of tools for the controlled manipulation of input stimuli and metrics for the output responses
evoked in a face matching system. This approach was inspired by Cambridge Face Memory Test [9]}, which involves progressively perturbing face images using a chosen transformation function (e.g. adding noise) to investigate controlled degradation of matching performance [5]}. This approach enables detailed studies of the failure modes of a face matcher or exploring how facial attributes are expressed in a network [6]}[7]}. In contrast, our approach generates controlled degradations using inpainting, to provide localized ground truth for evaluation of network attention models.
In [1]}, the authors propose a novel loss function to encourage part separability during network dissection of parts in a convolutional network for face matching. This approach is primarily concerned with training new networks to maximize interpretability, rather than studying existing networks. In [2]}, the authors study pairwise matching of faces, to visualize features that lead towards classification decisions. This is similar in spirit to our proposed approach, however we provide a performance metric for evaluating a saliency approach as well as extending visualizations to mated and nonmated triplets. Finally, in [4]}, the authors visualize the features of shape and texture that underlie subject identity decisions. This approach uses 3D modeling to generate a controlled dataset, rather than inpainting. However, given the authors conclusions that texture has a much larger effect of matching than morphology, having a ground truth dataset that includes texture variation would be an appropriate metric for explainable face recognition.
