The K-nearest neighbor (KNN) classifier is one of the simplest and most common classifiers, yet its performance competes with the most complex classifiers in the literature. The core of this classifier depends mainly on measuring the distance or similarity between the tested example and the training examples. This raises a major question about which distance measures to be used for the KNN classifier among a large number of distance and similarity measures? This review attempts to answer the previous question through evaluating the performance (measured by accuracy, precision and recall) of the KNN using a large number of distance measures, tested on a number of real world datasets, with and without adding different levels of noise. The experimental results show that the performance of KNN classifier depends significantly on the distance used, the results showed large gaps between the performances of different distances. We found that a recently proposed non-convex distance performed the best when applied on most datasets comparing to the other tested distances. In addition, the performance of the KNN degraded only about @math while the noise level reaches @math , this is true for all the distances used. This means that the KNN classifier using any of the top @math distances tolerate noise to a certain degree. Moreover, the results show that some distances are less affected by the added noise comparing to other distances.
We showed in this work how the Hassanat distance metric enhances the performance of the nearest neighb our classifiers. The results demonstrate the superiority of this distanc e metric over the traditional and most-used distanc es, such as Manhattan distance and Euclidian distance. Moreover, we prove d that the Hassanat distance metric is invariant to data scale, noise and outliers. Throughout this work, it is clearly notab le that both ENN and IINC performed very well with the distance investigated, as their accuracy increased significantly by 3.3 a nd 3.1 respectively, with no significant advantage of the ENN over the IINC in terms of accuracy. Correspondingly, it can be no ted from our results that there is no optimal algor ithm that can solve all reallife problems perfectly; this is supported by the n o-free-lunch theorem.