private DataSet [ ] splitDataSetUsingEM ( DataSet dataSet , int nrOfPartitions ) throws Exception { if ( dataSet . size ( ) <= 1 ) throw new Exception ( "EMsplit needs at least 2 objects!" ) ; EMProjectedClustering myEM = new EMProjectedClustering ( ) ; // iterate several times and take best solution int nrOfIterations = 1 ; // maximum --> 10 iterations // 10^2 objects --> 8 iterations // 10^6 objects --> 4 iteration // minimum --> 2 iterations // // #iterations = max{1, (10 - log_10(#objects)) } double log10 = Math . log ( dataSet . size ( ) * 1.0 ) / Math . log ( 10.0 ) ; nrOfIterations = Math . max ( 1 , ( 10 - ( ( Long ) Math . round ( log10 ) ) . intValue ( ) ) ) ; nrOfIterations = Math . min ( 10 , nrOfIterations ) ; int [ ] [ ] emMapping = myEM . getEMClusteringVariancesBestChoice ( dataSet . getFeaturesAsArray ( ) , nrOfPartitions , nrOfIterations ) ; DataSet [ ] subDataSets = new DataSet [ emMapping . length ] ; for ( int i = 0 ; i < subDataSets . length ; i ++ ) { subDataSets [ i ] = new DataSet ( dataSet . getNrOfDimensions ( ) ) ; for ( int j = 0 ; j < emMapping [ i ] . length ; j ++ ) { subDataSets [ i ] . addObject ( dataSet . getObject ( emMapping [ i ] [ j ] ) ) ; } } //////////////////////////////////////////////////////////////// // the EM part ends here // now we try to create at least 2 partitions // and make sure that each partition contains at least 2 objects //////////////////////////////////////////////////////////////// if ( subDataSets . length < 2 ) { System . out . println ( "mean shift split" ) ; subDataSets = splitDataSetUsingMeanShift ( dataSet ) ; } // decide what to do with kernels in inner nodes // by default they are allowed, i.e. no changes are made if the case occurs boolean changes = ! ALLOW_KERNELS_IN_INNER_NODES ; while ( changes ) { changes = false ; for ( int i = 0 ; i < subDataSets . length ; i ++ ) { if ( subDataSets [ i ] . size ( ) == 1 ) { System . out . println ( "merge singular sets" ) ; subDataSets = mergeDataSets ( subDataSets , i ) ; changes = true ; break ; } } } return subDataSets ; }
This methods splits the given data set into partitions using the EM algorithm . The resulting number of partitions is < = nrOfPartitions . If EM returns only one partition the data set is split using mean shift . The size of each resulting partition is AT LEAST 2 i . e . partitions with size 1 are merged with the closest remaining partition . Hence this method might return only one partition containing the whole data set .