@ SuppressWarnings ( "WeakerAccess" ) public static double compareChi2 ( TDigest dist1 , TDigest dist2 , double [ ] qCuts ) { double [ ] [ ] count = new double [ 2 ] [ ] ; count [ 0 ] = new double [ qCuts . length + 1 ] ; count [ 1 ] = new double [ qCuts . length + 1 ] ; double oldQ = 0 ; double oldQ2 = 0 ; for ( int i = 0 ; i <= qCuts . length ; i ++ ) { double newQ ; double x ; if ( i == qCuts . length ) { newQ = 1 ; x = Math . max ( dist1 . getMax ( ) , dist2 . getMax ( ) ) + 1 ; } else { newQ = qCuts [ i ] ; x = dist1 . quantile ( newQ ) ; } count [ 0 ] [ i ] = dist1 . size ( ) * ( newQ - oldQ ) ; double q2 = dist2 . cdf ( x ) ; count [ 1 ] [ i ] = dist2 . size ( ) * ( q2 - oldQ2 ) ; oldQ = newQ ; oldQ2 = q2 ; } return llr ( count ) ; }
Use a log - likelihood ratio test to compare two distributions . This is done by estimating counts in quantile ranges from each distribution and then comparing those counts using a multinomial test . The result should be asymptotically chi^2 distributed if the data comes from the same distribution but this isn t so much useful as a traditional test of a null hypothesis as it is just a reasonably well - behaved score that is bigger when the distributions are more different subject to having enough data to tell .