public double compare ( String v1 , String v2 ) { // FIXME: it should be possible here to say that, actually, we // didn't learn anything from comparing these two values, so that // probability is set to 0.5. if ( comparator == null ) return 0.5 ; // we ignore properties with no comparator // first, we call the comparator, to get a measure of how similar // these two values are. note that this is not the same as what we // are going to return, which is a probability. double sim = comparator . compare ( v1 , v2 ) ; // we have been configured with a high probability (for equal // values) and a low probability (for different values). given // sim, which is a measure of the similarity somewhere in between // equal and different, we now compute our estimate of the // probability. // if sim = 1.0, we return high. if sim = 0.0, we return low. for // values in between we need to compute a little. the obvious // formula to use would be (sim * (high - low)) + low, which // spreads the values out equally spaced between high and low. // however, if the similarity is higher than 0.5 we don't want to // consider this negative evidence, and so there's a threshold // there. also, users felt Duke was too eager to merge records, // and wanted probabilities to fall off faster with lower // probabilities, and so we square sim in order to achieve this. if ( sim >= 0.5 ) return ( ( high - 0.5 ) * ( sim * sim ) ) + 0.5 ; else return low ; }
Returns the probability that the records v1 and v2 came from represent the same entity based on high and low probability settings etc .