A High-Performance Customer Churn Prediction System based on Self-Attention
Customer churn prediction is a challenging domain of research that
contributes to customer retention strategy. The predictive performance of
existing machine learning models, which are often adopted by churn communities,
appear to be at a bottleneck, partly due to models' poor feature extraction
capability. Therefore, a novel algorithm, a hybrid neural network with
self-attention enhancement (HNNSAE), is proposed in this paper to improve the
efficiency of feature screening and feature extraction, consequently improving
the model's predictive performance. This model consists of three main blocks.
The first block is the entity embedding layer, which is employed to process the
categorical variables transformed into 0-1 code. The second block is the
feature extractor, which extracts the significant features through the
multi-head self-attention mechanism. In addition, to improve the feature
extraction effect, we stack the residual connection neural network on
multi-head self-attention modules. The third block is a classifier, which is a
three-layer multilayer perceptron. This work conducts experiments on publicly
available dataset related to commercial bank customers. The result demonstrates
that HNNSAE significantly outperforms the other Individual Machine Learning
(IML), Ensemble Machine Learning (EML), and Deep Learning (DL) methods tested
in this paper. Furthermore, we compare the performance of the feature extractor
proposed in this paper with that of other three feature extractors and find
that the method proposed in this paper significantly outperforms other methods.
In addition, four hypotheses about model prediction performance and overfitting
risk are tested on the publicly available dataset.