Spark based rare association rule mining for big datasets
Nowadays, the quality of wireless network becomes critical for the network service providers (NSP) A poor performed network may lead to customer complaints, even loss of revenue from user churns. To ensure the quality of service, the key quality indicators (KQI) which reflect the quality of specific use cases have been collected alongside the network performance counters (NPC) for network performance analysis. To start, the NSP mines the network performance data to discover KQI anomalies which may cause poor user experience. If there is any KQI anomaly has been detected, the NSP investigates the associated NPCs to identify the possible root causes. Since the number of use cases increases dramatically and the volume of collected network performance data grows tremendously everyday, the wireless network anomaly root cause analysis becomes extremely challenging. How to efficiently discover the relationship between NPCs and KQI outliers becomes the key to identify the root causes of anomalies in the wireless network. To solve this problem, in this paper, we propose an efficient rare association rule mining algorithm called Spark based Rare Association Rule Mining (SRAM) which leverages not only the efficiency of FP growth algorithm but also the powerful big data processing mechanism of Spark platform. We have implemented our algorithm on the Spark platform and tested with various of data sets. The result shows our method can efficiently mine rare association rules from big volume of data.
2016 IEEE International Conference on Big Data (Big Data)