Big Data Quality: A systematic literature review and future research directions <s> Comparison of Review papers <s> In the field of wireless sensor networks, those measurements that significantly deviate from the normal pattern of sensed data are considered as outliers. The potential sources of outliers include noise and errors, events, and malicious attacks on the network. Traditional outlier detection techniques are not directly applicable to wireless sensor networks due to the nature of sensor data and specific requirements and limitations of the wireless sensor networks. This survey provides a comprehensive overview of existing outlier detection techniques specifically developed for the wireless sensor networks. Additionally, it presents a technique-based taxonomy and a comparative table to be used as a guideline to select a technique suitable for the application at hand based on characteristics such as data type, outlier type, outlier identity, and outlier degree. <s> BIB001 </s> Big Data Quality: A systematic literature review and future research directions <s> Comparison of Review papers <s> Organizations are nowadays immersed in the Big Data Era. Beyond the hype of the concept of Big Data, it is true that something in the way of doing business is really changing. Although some challenges keep being the same as for regular data, with big data, the focus has changed. The reason is due to Big Data is not only data, but also a complete framework including data themselves, storage, formats, and ways of provisioning, processing and analytics. A challenge that becomes even trickier is the one concerning to the management of the quality of big data. More than ever the need for assessing the quality-in-use of big datasets gains importance since the real contribution – business value- of a dataset to a business can be only estimated in its context of use. Although there exists different data quality models to assess the quality of data there still lacks of a quality-in-use model adapted to big data. To fill this gap, and based on ISO 25012 and ISO 25024, we propose the 3Cs model, which is composed of three data quality dimensions for assessing the quality-in-use of big datasets: Contextual Consistency, Operational Consistency and Temporal Consistency. <s> BIB002 </s> Big Data Quality: A systematic literature review and future research directions <s> Comparison of Review papers <s> In our Big Data era, data is being generated, collected and analyzed at an unprecedented scale, and data-driven decision making is sweeping through all aspects of society. Recent studies have shown that poor quality data is prevalent in large databases and on the Web. Since poor quality data can have serious consequences on the results of data analyses, the importance of veracity, the fourth ‘V’ of big data is increasingly being recognized. In this tutorial, we highlight the substantial challenges that the first three ‘V’s, volume, velocity and variety, bring to dealing with veracity in big data. Due to the sheer volume and velocity of data, one needs to understand and (possibly) repair erroneous data in a scalable and timely manner. With the variety of data, often from a diversity of sources, data quality rules cannot be specified a priori; one needs to let the “data to speak for itself” in order to discover the semantics of the data. This tutorial presents recent results that are relevant to big data quality management, focusing on the two major dimensions of (i) discovering quality issues from the data itself, and (ii) trading-off accuracy vs efficiency, and identifies a range of open problems for the community. <s> BIB003 </s> Big Data Quality: A systematic literature review and future research directions <s> Comparison of Review papers <s> In the statistics community, outlier detection for time series data has been studied for decades. Recently, with advances in hardware and software technology, there has been a large body of work on temporal outlier detection from a computational perspective within the computer science community. In particular, advances in hardware technology have enabled the availability of various forms of temporal data collection mechanisms, and advances in software technology have enabled a variety of data management mechanisms. This has fueled the growth of different kinds of data sets such as data streams, spatio-temporal data, distributed streams, temporal networks, and time series data, generated by a multitude of applications. There arises a need for an organized and detailed study of the work done in the area of outlier detection with respect to such temporal datasets. In this survey, we provide a comprehensive and structured overview of a large set of interesting outlier definitions for various forms of temporal data, novel techniques, and application scenarios in which specific definitions and techniques have been widely used. <s> BIB004 </s> Big Data Quality: A systematic literature review and future research directions <s> Comparison of Review papers <s> A USAF sponsored MITRE research team undertook four separate, domain-specific case studies about Big Data applications. Those case studies were initial investigations into the question of whether or not data quality issues encountered in Big Data collections are substantially different in cause, manifestation, or detection than those data quality issues encountered in more traditionally sized data collections. The study addresses several factors affecting Big Data Quality at multiple levels, including collection, processing, and storage. Though not unexpected, the key findings of this study reinforce that the primary factors affecting Big Data reside in the limitations and complexities involved with handling Big Data while maintaining its integrity. These concerns are of a higher magnitude than the provenance of the data, the processing, and the tools used to prepare, manipulate, and store the data. Data quality is extremely important for all data analytics problems. From the study's findings, the "truth about Big Data" is there are no fundamentally new DQ issues in Big Data analytics projects. Some DQ issues exhibit return-s-to-scale effects, and become more or less pronounced in Big Data analytics, though. Big Data Quality varies from one type of Big Data to another and from one Big Data technology to another. <s> BIB005 </s> Big Data Quality: A systematic literature review and future research directions <s> Comparison of Review papers <s> Though the issues of data quality trace back their origin to the early days of computing, the recent emergence of Big Data has added more dimensions. Furthermore, given the range of Big Data applications, potential consequences of bad data quality can be for more disastrous and widespread. This paper provides a perspective on data quality issues in the Big Data context. it also discusses data integration issues that arise in biological databases and attendant data quality issues. <s> BIB006 </s> Big Data Quality: A systematic literature review and future research directions <s> Comparison of Review papers <s> This article investigates the evolution of data quality issues from traditional structured data managed in relational databases to Big Data. In particular, the paper examines the nature of the relationship between Data Quality and several research coordinates that are relevant in Big Data, such as the variety of data types, data sources and application domains, focusing on maps, semi-structured texts, linked open data, sensor & sensor networks and official statistics. Consequently a set of structural characteristics is identified and a systematization of the a posteriori correlation between them and quality dimensions is provided. Finally, Big Data quality issues are considered in a conceptual framework suitable to map the evolution of the quality paradigm according to three core coordinates that are significant in the context of the Big Data phenomenon: the data type considered, the source of data, and the application domain. Thus, the framework allows ascertaining the relevant changes in data quality emerging with the Big Data phenomenon, through an integrative and theoretical literature review. <s> BIB007 </s> Big Data Quality: A systematic literature review and future research directions <s> Comparison of Review papers <s> Data quality management systems are thoroughly researched topics and have resulted in many tools and techniques developed by both academia and industry. However, the advent of Big Data might pose some serious questions pertaining to the applicability of existing data quality concepts. There is a debate concerning the importance of data quality for Big Data; one school of thought argues that high data quality methods are essential for deriving higher level analytics while another school of thought argues that data quality level will not be so important as the volume of Big Data would be used to produce patterns and some amount of dirty data will not mask the analytic results which might be derived. This paper aims to investigate various components and activities forming part of data quality management such as dimensions, metrics, data quality rules, data profiling and data cleansing. The result list existing challenges and future research areas associated with Big Data for data quality management. <s> BIB008 </s> Big Data Quality: A systematic literature review and future research directions <s> Comparison of Review papers <s> High-quality data are the precondition for analyzing and using big data and for guaranteeing the value of the data. Currently, comprehensive analysis and research of quality standards and quality assessment methods for big data are lacking. First, this paper summarizes reviews of data quality research. Second, this paper analyzes the data characteristics of the big data environment, presents quality challenges faced by big data, and formulates a hierarchical data quality framework from the perspective of data users. This framework consists of big data quality dimensions, quality characteristics, and quality indexes. Finally, on the basis of this framework, this paper constructs a dynamic assessment process for data quality. This process has good expansibility and adaptability and can meet the needs of big data quality assessment. The research results enrich the theoretical scope of big data and lay a solid foundation for the future by establishing an assessment model and studying evaluation algorithms. <s> BIB009 </s> Big Data Quality: A systematic literature review and future research directions <s> Comparison of Review papers <s> Big Data refers to data volumes in the range of Exabyte (1018) and beyond. Such volumes exceed the capacity of current on-line storage and processing systems. With characteristics like volume, velocity and variety big data throws challenges to the traditional IT establishments. Computer assisted innovation, real time data analytics, customer-centric business intelligence, industry wide decision making and transparency are possible advantages, to mention few, of Big Data. There are many issues with Big Data that warrant quality assessment methods. The issues are pertaining to storage and transport, management, and processing. This paper throws light into the present state of quality issues related to Big Data. It provides valuable insights that can be used to leverage Big Data science activities. <s> BIB010 </s> Big Data Quality: A systematic literature review and future research directions <s> Comparison of Review papers <s> The massive growth in the scale of data has been observed in recent years being a key factor of the Big Data scenario. Big Data can be defined as high volume, velocity and variety of data that require a new high-performance processing. Addressing big data is a challenging and time-demanding task that requires a large computational infrastructure to ensure successful data processing and analysis. The presence of data preprocessing methods for data mining in big data is reviewed in this paper. The definition, characteristics, and categorization of data preprocessing approaches in big data are introduced. The connection between big data and data preprocessing throughout all families of methods and big data technologies are also examined, including a review of the state-of-the-art. In addition, research challenges are discussed, with focus on developments on different big data framework, such as Hadoop, Spark and Flink and the encouragement in devoting substantial research efforts in some families of data preprocessing methods and applications on new big data learning paradigms. <s> BIB011 </s> Big Data Quality: A systematic literature review and future research directions <s> Comparison of Review papers <s> Data Stream Management Systems (DSMS) provide real-time data processing in an effective way, but there is always a tradeoff between data quality (DQ) and performance. We propose an ontology-based data quality framework for relational DSMS that includes DQ measurement and monitoring in a transparent, modular, and flexible way. We follow a threefold approach that takes the characteristics of relational data stream management for DQ metrics into account. While (1) Query Metrics respect changes in data quality due to query operations, (2) Content Metrics allow the semantic evaluation of data in the streams. Finally, (3) Application Metrics allow easy user-defined computation of data quality values to account for application specifics. Additionally, a quality monitor allows us to observe data quality values and take counteractions to balance data quality and performance. The framework has been designed along a DQ management methodology suited for data streams. It has been evaluated in the domains of transportation systems and health monitoring. <s> BIB012 </s> Big Data Quality: A systematic literature review and future research directions <s> Comparison of Review papers <s> With the fast advance of big data technology and analytics solutions, big data computing and service is becoming a very hot research and application subject in academic research, industry community, and government services. Nevertheless, there are increasing data quality problems resulting in erroneous data costs in enterprises and businesses. Current research seldom discusses how to effectively validate big data to ensure data quality. This paper provides informative discussions for big data validation and quality assurance, including the essential concepts, focuses, and validation process. Moreover, the paper presents a comparison among big data validation tools and several major players in industry are discussed. Furthermore, the primary issues, challenges, and needs are discussed. <s> BIB013 </s> Big Data Quality: A systematic literature review and future research directions <s> Comparison of Review papers <s> With the rapid development of social networks, Internet of things, Cloud computing as well as other technologies, big data age is arriving. The increasing number of data has brought great value to the public and enterprises. Meanwhile how to manage and use big data better has become the focus of all walks of life. The 4V characteristics of big data have brought a lot of issues to the big data processing. The key to big data processing is to solve data quality issue, and to ensure data quality is a prerequisite for the successful application of big data technique. In this paper, we use recommendation systems and prediction systems as typical big data applications, and try to find out the data quality issues during data collection, data preprocessing, data storage and data analysis stages of big data processing. According to the elaboration and analysis of the proposed issues, the corresponding solutions are also put forward. Finally, some open problems to be solved in the future are also raised. <s> BIB014 </s> Big Data Quality: A systematic literature review and future research directions <s> Comparison of Review papers <s> A data stream is a sequence of data generated continuously over time. A data stream is too big to be saved in memory and its underlying data distribution may change over time. Outlier detection aims to find data instances which significantly deviate from the underlying data distribution. While outlier detection is performed at an individual instance level, anomalous pattern detection involves detecting a point in time where the behavior of the data becomes unusual and differs from normal behavior. Most outlier detection methods work in unsupervised mode, where the class labels of the data samples are not known. Alternatively, concept drift detection methods find a drifting point in the streaming data and try to adapt the model to the new emerging pattern. In this paper, we provide a review of outlier detection, anomaly pattern detection and concept drift detection approaches for streaming data. <s> BIB015 </s> Big Data Quality: A systematic literature review and future research directions <s> Comparison of Review papers <s> With the advances in communication technologies and the high amount of data generated, collected, and stored, it becomes crucial to manage the quality of this data deluge in an efficient and cost-effective way. The storage, processing, privacy and analytics are the main keys challenging aspects of Big Data that require quality evaluation and monitoring. Quality has been recognized by the Big Data community as an essential facet of its maturity. Yet, it is a crucial practice that should be implemented at the earlier stages of its lifecycle and progressively applied across the other key processes. The earlier we incorporate quality the full benefit we can get from insights. In this paper, we first identify the key challenges that necessitates quality evaluation. We then survey, classify and discuss the most recent work on Big Data management. Consequently, we propose an across-the-board quality management framework describing the key quality evaluation practices to be conducted through the different Big Data stages. The framework can be used to leverage the quality management and to provide a roadmap for Data scientists to better understand quality practices and highlight the importance of managing the quality. We finally, conclude the paper and point to some future research directions on quality of Big Data. <s> BIB016
In this section, review papers are compared according to different criteria. Table 6 compares all review papers mentioned in Sections 7.1 and 7.2, with the proposed systematic review. BIB002 2015 Discussion N/A 57 Batch Evaluation N/A General BIB010 2016 Discussion N/A 8 Batch Evaluation N/A General BIB011 2016 Review 32 BIB012 Batch Cleaning Technique based General  2016 Discussion N/A 3 Batch Evaluation N/A General BIB013 2016 Discussion N/A 30 Batch Evaluation N/A General BIB005 2015 Discussion N/A 23 Batch Evaluation N/A General BIB014 2017 Discussion N/A 1 Batch Evaluation N/A General BIB006 2015 Discussion N/A 8 Batch Evaluation N/A General BIB003 2014 Discussion N/A 115 Batch Evaluation N/A General BIB007 2015 Discussion N/A 33 Batch Evaluation N/A General BIB008 2015 Discussion N/A 21 Batch Evaluation N/A General BIB009 2015 The date column refers to the publication date of a paper, which by observing the values of this column, we find that the publication date of studies  , BIB015 , BIB016 is 2018, but these papers do not cover the studies of this year. Paper  , mentions the researches until 2013, paper BIB015 discuss the studies published until 2016, and finally, paper BIB016 cover the publications until 2017, and the proposed method is the only paper that covers topics of 2018. The review type column specifies the type of each publication that, if it is a systematic literature review, is named SLR, also, if the review paper is non-systematic, is called Review, and if it only mentions the challenges, Discussion is considered in the table. The next column refers to a number of studies, which each paper describes that according to the table, just paper BIB004 explains further studies than the proposed paper. In our method, each of the 88 studies is described individually, but in BIB004 , sometimes more than ten references are included in a paragraph. Number of citations is another important criterion for recognizing the goodness of a study. By examining the values of this column, we find that papers BIB004 , BIB001 , BIB009 , BIB003 have the highest number of citations, but none of them has covered the recent year studies. Processing type is another criterion to evaluate review papers, which if a study only considers stream processing, then Stream is in the table, moreover, if the focus is on static data, Batch is considered, and if the method reviews hybrid studies, Hybrid is written in the table. As indicated in the table, the proposed method is the only paper that considers all three types of processing. The next column is Task, and as previously stated, there are three tasks for improving the data quality which is: Outlier Detection, Evaluation, and Cleaning. Since some papers also present anomaly detection methods in addition to the outlier detection, which as stated these methods do not improve data quality and only detect an abnormal pattern to prevent system attack, we considered Detection, instead of Outlier detection, for these type of studies. As is evident, the method presented in this study is the only paper that considers all big data quality tasks. Classification perspective is another evaluation criterion, which determines whether any classifications for the methods is presented in the paper or not and if presented, what kind of classification is provided? Is classification based on the processing type, technique-based or task-based? The discussion studies do not provide any classifications on research papers; on the other hand, some Review papers also explain the related methods without any classifications. The proposed method provides a new classification compared to other overview review papers. Finally, the last column is the Domain Application, which means that if the domain of paper is not explicitly stated, it is General and otherwise, it is mentioned in the table. Most studies do not specify a domain, and therefore the "general" is added for them.