Detection and prediction of insider threats to cyber security: a systematic literature review and meta-analysis <s> Comparative analysis of various IDPAs <s> Insider cyber threat is a serious problem in resent years. Many traditional methods such as intrusion detection system and prevention system can not effectively deal with insider attack problems because they lack of dynamic inference capability to acquire and understand cyber situational awareness. This paper presented a framework model based on DBN to capture the dynamic user behavior and establish and improve inference ability. This model has used transition relationship of DBN and HMM and its better performance inference algorithm to infer next activity. Those performances are verified and compared by the experiments in the end. <s> BIB001 </s> Detection and prediction of insider threats to cyber security: a systematic literature review and meta-analysis <s> Comparative analysis of various IDPAs <s> The ability to mine data represented as a graph has become important in several domains for detecting various structural patterns. One important area of data mining is anomaly detection, but little work has been done in terms of detecting anomalies in graph-based data. In this paper we present graph-based approaches to uncovering anomalies in applications containing information representing possible insider threat activity: e-mail, cell-phone calls, and order processing. <s> BIB002 </s> Detection and prediction of insider threats to cyber security: a systematic literature review and meta-analysis <s> Comparative analysis of various IDPAs <s> Collaborative information systems (CISs) are deployed within a diverse array of environments that manage sensitive information. Current security mechanisms detect insider threats, but they are ill-suited to monitor systems in which users function in dynamic teams. In this paper, we introduce the community anomaly detection system (CADS), an unsupervised learning framework to detect insider threats based on the access logs of collaborative environments. The framework is based on the observation that typical CIS users tend to form community structures based on the subjects accessed (e.g., patients' records viewed by healthcare providers). CADS consists of two components: 1) relational pattern extraction, which derives community structures and 2) anomaly prediction, which leverages a statistical model to determine when users have sufficiently deviated from communities. We further extend CADS into MetaCADS to account for the semantics of subjects (e.g., patients' diagnoses). To empirically evaluate the framework, we perform an assessment with three months of access logs from a real electronic health record (EHR) system in a large medical center. The results illustrate our models exhibit significant performance gains over state-of-the-art competitors. When the number of illicit users is low, MetaCADS is the best model, but as the number grows, commonly accessed semantics lead to hiding in a crowd, such that CADS is more prudent. <s> BIB003 </s> Detection and prediction of insider threats to cyber security: a systematic literature review and meta-analysis <s> Comparative analysis of various IDPAs <s> The annual incidence of insider attacks continues to grow, and there are indications this trend will continue. While there are a number of existing tools that can accurately identify known attacks, these are reactive (as opposed to proactive) in their enforcement, and may be eluded by previously unseen, adversarial behaviors. This paper proposes an approach that combines Structural Anomaly Detection (SA) from social and information networks and Psychological Profiling (PP) of individuals. SA uses technologies including graph analysis, dynamic tracking, and machine learning to detect structural anomalies in large-scale information network data, while PP constructs dynamic psychological profiles from behavioral patterns. Threats are finally identified through a fusion and ranking of outcomes from SA and PP. The proposed approach is illustrated by applying it to a large data set from a massively multi-player online game, World of War craft (WoW). The data set contains behavior traces from over 350,000 characters observed over a period of 6 months. SA is used to predict if and when characters quit their guild (a player association with similarities to a club or workgroup in non-gaming contexts), possibly causing damage to these social groups. PP serves to estimate the five-factor personality model for all characters. Both threads show good results on the gaming data set and thus validate the proposed approach. <s> BIB004 </s> Detection and prediction of insider threats to cyber security: a systematic literature review and meta-analysis <s> Comparative analysis of various IDPAs <s> Evidence of malicious insider activity is often buried within large data streams, such as system logs accumulated over months or years. Ensemble-based stream mining leverages multiple classification models to achieve highly accurate anomaly detection in such streams, even when the stream is unbounded, evolving, and unlabeled. This makes the approach effective for identifying insider threats who attempt to conceal their activities by varying their behaviors over time. This paper applies ensemble-based stream mining, supervised and unsupervised learning, and graph-based anomaly detection to the problem of insider threat detection. It demonstrates that the ensemble-based approach is significantly more effective than traditional single-model methods, supervised learning outperforms unsupervised learning, and increasing the cost of false negatives correlates to higher accuracy. Future work will consider a wider range of tunable parameters in an effort to further reduce false positives, include a more sophisticated polling algorithm for weighting better models, and implement parallelization to lower runtimes to more rapidly detect emerging insider threats. <s> BIB005 </s> Detection and prediction of insider threats to cyber security: a systematic literature review and meta-analysis <s> Comparative analysis of various IDPAs <s> This paper introduces a Bayesian network model for the motivation and psychology of the malicious insider. First, an initial model was developed based on results in the research literature, highlighting critical variables for the prediction of degree of interest in a potentially malicious insider. Second, a survey was conducted to measure these predictive variables in a common sample of normal participants. Third, a structural equation model was constructed based on the original model, updated based on a split-half sample of the empirical survey data and validated against the other half of the dataset. Fourth, the Bayesian network was adjusted in light of the results of the empirical analysis. Fifth, the updated model was used to develop an upper bound on the quality of model predictions of its own simulated data. When empirical data regarding psychological predictors were input to the model, predictions of counterproductive behavior approached the upper bound of model predictiveness. <s> BIB006 </s> Detection and prediction of insider threats to cyber security: a systematic literature review and meta-analysis <s> Comparative analysis of various IDPAs <s> In recent years, the major source of information leakage is due to insiders. In order to detect information leakage by some internal insiders, anomaly detection using individual and community behavior models have been developed. The basic assumption of anomaly detection is each user has his/her own profile of activities and anomaly detection algorithm attempts to identify any deviation from the basic profile by each user. Both models neglected the possibility of change of individual user profile, e.g. change of individual interests. We propose here an anomaly detection model of insider threat using file content. The proposed model uses the document segmentation and Naive Bayes algorithm to classify the contents of files in an organization. We then set up the correlation matrices between users and their interests, and also the user community and their interests. We then propose a comprehensive model to detect the insider threat, which takes into consideration of the deviations of individual users’ current behaviors, their historical behaviors and their associated community behaviors simultaneously. According to the experimental test results, the proposed model can successfully detect the anomaly access to files in the internal systems. <s> BIB007 </s> Detection and prediction of insider threats to cyber security: a systematic literature review and meta-analysis <s> Comparative analysis of various IDPAs <s> Abstract Insider threat is one of the most dangerous security threat, and a much more complex issue. These insiders can be a former or a disgruntled employee or any business associate that has or had an authorised access to information for any particular organization. They have control and security measures. Hence continuous monitoring is essential to track each and every activity within the network. Log management is a strong technique which includes both Log analysis with event correlation which provides the root cause of any attack and network can be protected from security violations. Though intrusion detection is complex process, while checking the ability to detect intrusive behaviour within the internal environment, it has to take care of suppressing the false alarm rate. Some strong approach is required on the basis of which decisions can be taken fast. This paper proposes a probabilistic approach which illustrates the frequency of occurrence of event in percentage while still considering the false alarm rate at an acceptable level. <s> BIB008
Not all of the selected papers propose novel algorithms. Moreover, many of the proposed algorithms have never been implemented. In only 13 papers out of the 37 selected papers, authors have actually implemented and evaluated their proposed algorithms. Hence, we selected these 13 papers for the meta-analysis [12-15, 18, 21-27, 29] . Figure 9 depicts the total number of citations received by all these 13 papers based on Google Scholar's citation count. We also looked at the papers that cited these 13 selected research papers. However, none of these citation papers use the IDPAs presented in these research papers. The most cited papers are: Chen et al. BIB003 and Brdiczka et al. BIB004 . Each of the 13 selected papers proposed one IDPA except for one paper (Parveen et al. BIB005 ). In this paper, Parveen et al. BIB005 present two IDPAs. These algorithms are-GBAD and OCSVM. In the rest of the paper, we refer GBAD by 'Parveen et al. BIB005 : GBAD' and OCSVM by 'Parveen et al. BIB005 : OCSVM'. An overview of all these 14 algorithms are provided in Additional file 1: Table S1 . The authors of these 14 algorithms have evaluated their proposed algorithms on different datasets. The majority of the studies surveyed are evaluated on large datasets. Since they mainly use synthetic data or real data with only benign activities, the size of the data is not a problem in this case. The noise level is the prime factor to determine the optimal data size; and noise levels vary from data to data. Therefore, no comparisons can be made among different algorithms and studies in terms of the size of the datasets used. In addition these algorithms are evaluated on the predictive performance using different metrics. For example, Axelrad et al. BIB006 measure performance in terms of error rate, logarithmic loss and quadratic loss. Zhang et al. BIB007 and Ambre & Shekokar BIB008 do not offer a predicting performance assessment. Tang, Zhou and Wang BIB001 measure performance in terms of inference and learning time. Our goal is to compare all the algorithms using the true positive rate (detection rate) and the false positive rate (false alarm rate). Detection rates (DRs) and false alarm rates (FARs) have been derived from the reported performance values that are found in the individual papers. We are able to convert the performance values published in nine studies [12-14, 21-24, 27, 29] (out of the 13 studies) where levels of information are sufficient to calculate DR and FAR. The estimated DRs and FARs of ten IDPAs (from nine studies) are shown in Additional file 1: Table S1 and Fig. 10 . Figure 11 displays the Euclidean distance of the performance of each of the ten algorithms from the desired performance metrics (DR: 100 % and FAR: 0 %). Since the IDPAs are evaluated on different datasets, we can't compare their empirical performance. However these empirical results do provide two interesting observations: First, OCSVM and GBAD proposed by Parveen et al. BIB005 were evaluated on the same dataset and GBAD's performance was much worse than that of OCSVM. Fig. 9 Number of citations for each selected publication based on the Google Scholar. Only those publications are displayed where the authors implemented their proposed algorithms and the evaluation are presented. The horizontal x-axis represents the number of citations for publications for the author on the vertical y-axis Second, Eberie et al. BIB002 propose a similar algorithm (they also call their algorithm 'GBAD') and their algorithm performed extremely well on the real-world datasets. Their difference is that Parveeen's GBAD is an ensemble comprising k number of BNs compared to Eberie's GBAD which is a single BN model. Parveen's GBAD is a complex algorithm. Learning an optimal BN is a NP-hard problem and Parveen's GBAD consists of multiple BNs. In fact it is possible that the authors would not find the optimal model solution. Also, the way the outputs of the ensemble members are combined in Parveen's GBAD is questionable as they perform simple majority voting or weighted voting to reach consensus. These linear approaches cannot account for the interaction effects. Table S1 does not report the learning time of the algorithms, since among 14 algorithms only one algorithm was evaluated in terms of the learning time. However, the table shows the number of free parameters for the chosen 14 IDPAs. The learning time is proportional to the number of free parameters in the model. The higher the number of free parameters, the longer the learning time.