Statistical Significance of Ranking Paradoxes
When nonparametric statistical tests are used to rank-order a list of alternatives, Simpson-like paradoxes arise, in which the individual parts give rise to a common decision, but the aggregate of those parts gives rise to a different decision. Haunsperger (2003) and Bargagliotti (2009) showed that the Kruskal-Wallis (Kruskal and Wallis, 1952), Mann-Whitney (Mann and Whitney, 1947), and Bhapkar's V (Bhapkar, 1961) nonparametric statistical tests are subject to these types of paradoxes. We further investigate these ranking paradoxes by showing that when they occur, the differences in rankings are not statistically significant.