Big Data Quality: A systematic literature review and future research directions <s> Hybrid Methods <s> Big Data is often referred to as the 3Vs: Volume, Velocity and Variety. A 4th V (validity) was introduced to address the quality dimension. Poor data quality can be costly, lead to breaks in processes and invalidate the company's efforts on regulatory compliance. In order to process data streams in real time, a new technology called CEP (complex event processing) was developed. In France, the current deployment of smart meters will generate massive electricity consumption data. In this work, we developed a diagnostic approach to compute generic quality indicators of smart meter data streams on the fly. This solution is based on Tibco StreamBase CEP. Visualization tools were also developed in order to give a better understanding of the inter-relation between quality issues and geographical/temporal dimensions. According to the application purpose, two visualization methods can be loaded: (1) StreamBase LiveView is used to visualize quality indicators in real time; and (2) a Web application provides a posteriori and geographical analysis of the quality indicators which are plotted on a map within a color scale (lighter colors indicate good quality and darker colors indicate poor quality). In future works, new quality indicators could be added to the solution which can be applied in an operational context in order to monitor data quality from smart meters. <s> BIB001 </s> Big Data Quality: A systematic literature review and future research directions <s> Hybrid Methods <s> Abstract The use of sensors has had an enormous increment in the last years, becoming a valuable tool in many different areas. In this kind of scenario, the quality of data becomes an extremely important issue; however, not much attention has been paid to this specific topic, with only a few existing works that focus on it. In this paper, we present a proposal for managing data streams from sensors that are installed in patients’ homes in order to monitor their health. It focuses on processing the sensors’ data streams, taking into account data quality. In order to achieve this, a data quality model for this kind of data streams and an architecture for the monitoring system are proposed. Moreover, our work introduces a mechanism for avoiding false alarms generated by data quality problems. <s> BIB002 </s> Big Data Quality: A systematic literature review and future research directions <s> Hybrid Methods <s> AbstractWireless Sensor Networks (WSNs) are widely used for monitoring physical happenings of the environment. However, the data gathered by the WSNs may be inaccurate and unreliable due to power exhaustion, noise and other reasons. Unnecessary data such as erroneous data and redundant data transmission causes a lot of extra energy consumption. To improve the data reliability and reduce the energy consumption, we proposed an in-network processing architecture for data cleaning, which divides the task into four stages implemented in different nodes respectively. This strategy guaranteed the cleaning algorithms were computationally lightweight in local nodes and energy-efficient due to almost no communication overhead. In addition, we presented the detection algorithms for data faults and event outliers, which were conducted by utilizing the related attributes from the local sensor node and the cooperation with its relaying neighbor. Experiment results show that our proposed approach is accurate and energy-... <s> BIB003 </s> Big Data Quality: A systematic literature review and future research directions <s> Hybrid Methods <s> With the evolution of data and increasing popularity of IoT (Internet of Things), stream data mining has gained immense popularity. Researchers and developers are trying to analyze data patterns obtained from various devices. Stream data have several characteristics, the most important being its huge volume and high velocity. Although, a lot of research is being conducted in order to develop more efficient stream data mining techniques, pre-processing of stream data is an area that is under-studied. Real time applications generate data which is rather noisy and contain missing values. Apart from this, there is the issue of data evolution, which is a concern when dealing with stream data. To deal with the evolution of data, the proposed solution offers a hybrid of preprocessing techniques which are adaptive in nature. As a result of the study, an adaptive preprocessing and learning approach is implemented. The case study with sensor weather data demonstrates the results and accuracy of the proposed solution. <s> BIB004 </s> Big Data Quality: A systematic literature review and future research directions <s> Hybrid Methods <s> Abstract Context: Big Data systems are a class of software systems that ingest, store, process and serve massive amounts of heterogeneous data, from multiple sources. Despite their undisputed impact in current society, their engineering is still in its infancy and companies find it difficult to adopt them due to their inherent complexity. Existing attempts to provide architectural guidelines for their engineering fail to take into account important Big Data characteristics, such as the management, evolution and quality of the data. Objective: In this paper, we follow software engineering principles to refine the λ -architecture, a reference model for Big Data systems, and use it as seed to create Bolster , a software reference architecture (SRA) for semantic-aware Big Data systems. Method: By including a new layer into the λ -architecture, the Semantic Layer, Bolster is capable of handling the most representative Big Data characteristics (i.e., Volume, Velocity, Variety, Variability and Veracity). Results: We present the successful implementation of Bolster in three industrial projects, involving five organizations. The validation results show high level of agreement among practitioners from all organizations with respect to standard quality factors. Conclusion: As an SRA, Bolster allows organizations to design concrete architectures tailored to their specific needs. A distinguishing feature is that it provides semantic-awareness in Big Data Systems. These are Big Data system implementations that have components to simplify data definition and exploitation. In particular, they leverage metadata (i.e., data describing data) to enable (partial) automation of data exploitation and to aid the user in their decision making processes. This simplification supports the differentiation of responsibilities into cohesive roles enhancing data governance. <s> BIB005
Some methods work both on batch and stream data, which usually use static data to build a model and apply the model to the data stream. Sometimes authors run a specific algorithm on both types of data. Because there are no review papers, as shown in Figure 9 , these methods are divided into three categories: Outlier Detection, Evaluation, and Cleaning. Since the number of papers in hybrid methods is limited, there is no other classification on each category. The list of the papers based on the technique in hybrid methods is determined in Table 5 . BIB004 Rule-based BIB001 ,  [80] Model-based BIB002 [82], BIB003 Schema based BIB005 [85] The methods of each category are explained in the following sub-sections.