A review on the public benchmark databases for static keystroke dynamics <s> The Performance of a Dataset <s> Biometrics based user authentication involves collecting user's patterns and then using them to determine if a new pattern is similar enough. The quality of the user's patterns is as important as the quality of the classifier. But, the issue has been ignored in the literature since the popular biometrics are mostly trait based such as finger prints and iris so that its pattern quality depends on the quality of the input device involved. However, the quality of the user's patterns of behavior based biometric such as keystroke dynamics can be improved artificially by increasing the peculiarity of the typing style. In this paper, we propose several ways to improve the quality. But, first we define the quality of patterns in terms of two factors: uniqueness and consistency. Finally, the results of a preliminary experiment are presented that support the utility of the proposed methods. <s> BIB001 </s> A review on the public benchmark databases for static keystroke dynamics <s> The Performance of a Dataset <s> Keystroke dynamics is a behavioral biometrics showing a degradation of performance when used over time. This is due to the fact that the user improves his/her way of typing while using the system, therefore the test samples may be different from the initial template computed at an earlier stage. One way to bypass this problem is to use template update mechanisms. We propose in this work, new semi-supervised update mechanisms, inspired from known supervised ones. These schemes rely on the choice of two thresholds (an acceptance threshold and an update threshold) which are fixed manually depending on the performance of the system and the level of tolerance in possible inclusion of impostor data in the update template. We also propose a new evaluation scheme for update mechanisms, taking into account performance evolution over several time-sessions. Our results show an improvement of 50% in the supervised scheme and of 45% in the semi-supervised one with a configuration of the parameters chosen so that we do not accept many erroneous data. <s> BIB002
To compute performance figures, it is necessary to select a certain amount of data for enrollment and another amount for testing. As all the datasets are different, it is almost impossible to use the same number of training and validating samples across datasets. A good way to ease the comparison with few bias is to use the first session for the enrollment samples and the other sessions for the validating ones (we have chosen this procedure by keeping 50% of samples for training and 50% for testing for datasets having no sessions). We can see that comparison will be greatly biased between datasets having a huge difference in the amount of samples per session; but we had to make a choice. An even better procedure would be to split the dataset in three sub-datasets, forget the first one by considering it consists of user's practicing while the two other parts serve for training and testing ; however it would reduce the amount of samples available for training and testing, which is quite problematic as most of the datasets are quite small. The information provided by this category is related to the difficulty of the dataset. • Uniqueness. It is based on the distance of impostors samples against enrollment ones (higher is better). • Inconsistency. It is based on the similarity between query samples against enrolled ones and depends on concentration and dexterity of the user (lower is better). • Discriminability. It is based on the distance between the closest impostor sample to the mean of the genuine samples and the further genuine sample (higher is better). Even if these three indicators have been published in 2006 BIB001 , to our knowledge, nobody but their creators have ever used them. This information can be provided with (a) a list of scores, (b) its histogram, or (c) a mean value among the users. They may also depend on time (to our knowledge, it has never been verified). So, it may be useful to also compute them session per session and compute their mean value. Performance of a baseline classifier. The aim of KDB is to compare performance of various classifiers among various conditions. It is therefore important to give the performance of a baseline classifier using the dataset. The indicators can be the EER and the Area Under the Curve (AUC). The ROC Curve by itself is hardly interpretable by an automatic comparison tool. The EER gives a functional point often used in the literature to compare classifiers even if it is not an operational functional point of the algorithm (lower is better). The AUC is the area under the ROC curve and gives a more global performance information on the classifier (higher is better). As we characterize a dataset and not a recognition method, we do not think it is necessary to use more metrics. However, it would even be better to compute a confidence interval of the EER, because it can be quite wide. Note that this performance measure depends on the chosen baseline algorithm. Performance degradation over time. It is interesting to know if a dataset is more subject to performance degradation over time BIB002 by (i) computing the performance (EER) of each session using the samples of the first one for enrollment; a,d (ii) computing the slope of the regression line of the EER among the sessions. The larger the slope is, the larger the degradation over time. We consider all these criteria to compare the six benchmark datasets in the literature in the following section.