Big Data Quality: A systematic literature review and future research directions <s> Stream Review Papers <s> In the field of wireless sensor networks, those measurements that significantly deviate from the normal pattern of sensed data are considered as outliers. The potential sources of outliers include noise and errors, events, and malicious attacks on the network. Traditional outlier detection techniques are not directly applicable to wireless sensor networks due to the nature of sensor data and specific requirements and limitations of the wireless sensor networks. This survey provides a comprehensive overview of existing outlier detection techniques specifically developed for the wireless sensor networks. Additionally, it presents a technique-based taxonomy and a comparative table to be used as a guideline to select a technique suitable for the application at hand based on characteristics such as data type, outlier type, outlier identity, and outlier degree. <s> BIB001 </s> Big Data Quality: A systematic literature review and future research directions <s> Stream Review Papers <s> Wireless Sensor Networks (WSNs) are important and necessary platforms for the future as the concept “Internet of Things” has emerged lately. They are used for monitoring, tracking, or controlling of many applications in industry, health care, habitat, and military. However, the quality of data collected by sensor nodes is affected by anomalies that occur due to various reasons, such as node failures, reading errors, unusual events, and malicious attacks. Therefore, anomaly detection is a necessary process to ensure the quality of sensor data before it is utilized for making decisions. In this review, we present the challenges of anomaly detection in WSNs and state the requirements to design efficient and effective anomaly detection models. We then review the latest advancements of data anomaly detection research in WSNs and classify current detection approaches in five main classes based on the detection methods used to design these approaches. Varieties of the state-of-the-art models for each class are covered and their limitations are highlighted to provide ideas for potential future works. Furthermore, the reviewed approaches are compared and evaluated based on how well they meet the stated requirements. Finally, the general limitations of current approaches are mentioned and further research opportunities are suggested and discussed. <s> BIB002 </s> Big Data Quality: A systematic literature review and future research directions <s> Stream Review Papers <s> In the statistics community, outlier detection for time series data has been studied for decades. Recently, with advances in hardware and software technology, there has been a large body of work on temporal outlier detection from a computational perspective within the computer science community. In particular, advances in hardware technology have enabled the availability of various forms of temporal data collection mechanisms, and advances in software technology have enabled a variety of data management mechanisms. This has fueled the growth of different kinds of data sets such as data streams, spatio-temporal data, distributed streams, temporal networks, and time series data, generated by a multitude of applications. There arises a need for an organized and detailed study of the work done in the area of outlier detection with respect to such temporal datasets. In this survey, we provide a comprehensive and structured overview of a large set of interesting outlier definitions for various forms of temporal data, novel techniques, and application scenarios in which specific definitions and techniques have been widely used. <s> BIB003 </s> Big Data Quality: A systematic literature review and future research directions <s> Stream Review Papers <s> Data mining is one of the most exciting fields of research for the researcher. As data is getting digitized, systems are getting connected and integrated, scope of data generation and analytics has increased exponentially. Today, most of the systems generate non-stationary data of huge, size, volume, occurrence speed, fast changing etc. these kinds of data are called data streams. One of the most recent trend i.e. IOT (Internet Of Things) is also promising lots of expectation of people which will ease the use of day to day activities and it could also connect systems and people together. This situation will also lead to generation of data streams, thus present and future scope of data stream mining is highly promising. Characteristics of data stream possess many challenges for the researcher; this makes analytics of such data difficult and also acts as source of inspiration for researcher. Outlier detection plays important role in any application. In this paper we reviewed different techniques of outlier detection for stream data and their issues in detail and presented results of the same. <s> BIB004 </s> Big Data Quality: A systematic literature review and future research directions <s> Stream Review Papers <s> Real time data analysis in data streams is a highly challenging area in big data. The surge in big data techniques has recently attracted considerable interest to the detection of significant changes or anomalies in data streams. There is a variety of literature across a number of fields relevant to anomaly detection. The growing number of techniques, from seemingly disconnected areas, prevents a comprehensive review. Many interesting techniques may therefore remain largely unknown to the anomaly detection community at large. The survey presents a compact, but comprehensive overview of diverse strategies for anomaly detection in evolving data streams. A number of recommendations based performance and applicability to use cases are provided. We expect that our classification and recommendations will provide useful guidelines to practitioners in this rapidly evolving field. <s> BIB005 </s> Big Data Quality: A systematic literature review and future research directions <s> Stream Review Papers <s> ABSTRACTNowadays technological advances have promoted big data streams common in many applications, including mobile internet applications, internet of things, and industry production process. Outl... <s> BIB006 </s> Big Data Quality: A systematic literature review and future research directions <s> Stream Review Papers <s> A data stream is a sequence of data generated continuously over time. A data stream is too big to be saved in memory and its underlying data distribution may change over time. Outlier detection aims to find data instances which significantly deviate from the underlying data distribution. While outlier detection is performed at an individual instance level, anomalous pattern detection involves detecting a point in time where the behavior of the data becomes unusual and differs from normal behavior. Most outlier detection methods work in unsupervised mode, where the class labels of the data samples are not known. Alternatively, concept drift detection methods find a drifting point in the streaming data and try to adapt the model to the new emerging pattern. In this paper, we provide a review of outlier detection, anomaly pattern detection and concept drift detection approaches for streaming data. <s> BIB007
In this section, papers that have not provided the implemented method and only reviewed the past work or presented challenges and solutions are mentioned. The focus of most of the papers presented in this category is based on detecting outliers, and different review papers have provided a different categorization for outlier detection in the data stream. In  outlier detection methods are divided into five categories: statistical, distance based, density based, sliding window based and clustering methods. In the following, some challenges in streaming outlier detection are mentioned which are: transient, notion of a timestamp, infinite, arrival rate, concept drift, and uncertainty. The classification given in BIB006 has four categories and does not have a window-based category than the  . In  , BIB004 , the techniques in the data stream are divided into two groups based on clustering and outlier detection algorithms. Some available algorithms have been evaluated regarding the percentage of detection and the amount of memory usage and runtime. Challenges are divided into two groups of a single data stream and multiple data streams. In single data stream, all the challenges of  are mentioned and also multidimensional is added, then challenges in multiple data streams are cross-correlation, asynchronous data points, dynamic relationship, and heterogeneous schema. In BIB004 , a hybrid method is also proposed. A different categorization is presented in BIB005 . In this paper, in addition to the statistical, clustering, classification and similarity-based methods, there are also soft computing, knowledge-based and combination learners. Another similar classification is presented in  , in which statistics, clustering, classification, artificial intelligence, and the nearest neighbor are in this classification. In this paper, there are some advantages and disadvantages of each category. It is said that statistical methods are efficient for single-variable data and cannot find the relationship between the multivariate data. The nearest neighbor based methods are simple, but it is costly to calculate the distance of multivariate data. Also, clustering algorithms are suitable for incremental environments and classification based methods are more expensive than other methods to calculate the corresponding criteria. The classification of outlier detection methods of the paper BIB001 is very similar to  , there are spectral decomposition-based approaches instead of AI-based methods. In this category, normal data is obtained by the PCA algorithm. In BIB007 a brief review of outlier detection, anomaly detection and concept drift detection for data streams is presented. A different categorization is presented in BIB003 , which is classified according to the type of data. Data types are time-series, stream, distributed, spatial-temporal and network. If the data type is a data stream, methods are classified into model-based, distance based and highdimensional methods. In BIB002 first, the anomaly detection requirements in the sensor network are described, where data reduction, online detection, distributed detection, adaptive detection, and correlation exploitation are part of requirements. Then there are challenges to detect anomalies in the sensor network, such as resource constraints, communication overhead, dynamic network topology, and so forth. Besides, methods classification is provided for detecting anomalies in the sensor network, which includes statistics based, nearest neighbor based, clustering based and classification based methods. The advantages and disadvantages of each one are mentioned, and they are the same as  . In  , first, the characteristics of data and data quality dimensions on the Internet of things are pointed out, then the impact of important factors in reducing the quality of data on these dimensions has been investigated and ultimately, data quality assessment methods are described from the perspective of the data cleaning and methods have been compared in terms of technique, scope, data stream type and data characteristics.