1. A method, comprising: storing a plurality of removable facial feature templates and a plurality of non-removable facial feature templates; weighting each of the plurality of removable facial feature templates and the plurality of non-removable facial feature templates so that facial feature templates that are erroneously classified by a given classifier are more likely to be selected for further training than facial feature templates that were correctly classified; capturing, by a computing device, an image including at least a face of a user; calculating, by the computing device, a face template of the face of the user in the image; determining, by the computing device, whether the face template includes a removable facial feature that decreases a level of distinctiveness between two faces based at least in part on a weighted sum of similarity scores between the face template and the each of the plurality of removable facial feature templates; determining, by the computing device, whether the face template includes a non-removable facial feature that decreases a level of distinctiveness between two faces based at least in part on a weighted sum of similarity scores between the face template and the each of the plurality of non-removable facial feature templates; responsive to determining that the face includes the removable facial feature, outputting, by the computing device, a notification to request that the user remove the removable facial feature; and responsive to determining that the face includes the non-removable facial feature, adjusting, by the computing device, a first similarity score threshold to a second similarity score threshold.