Big Data Quality: A systematic literature review and future research directions <s> Rule-based techniques <s> With the abundance of raw data generated from various sources, Big Data has become a preeminent approach in acquiring, processing, and analyzing large amounts of heterogeneous data to derive valuable evidences. The size, speed, and formats in which data is generated and processed affect the overall quality of information. Therefore, Quality of Big Data (QBD) has become an important factor to ensure that the quality of data is maintained at all Big data processing phases. This paper addresses the QBD at the pre-processing phase, which includes sub-processes like cleansing, integration, filtering, and normalization. We propose a QBD model incorporating processes to support Data quality profile selection and adaptation. In addition, it tracks and registers on a data provenance repository the effect of every data transformation happened in the pre-processing phase. We evaluate the data quality selection module using large EEG dataset. The obtained results illustrate the importance of addressing QBD at an early phase of Big Data processing lifecycle since it significantly save on costs and perform accurate data analysis. <s> BIB001 </s> Big Data Quality: A systematic literature review and future research directions <s> Rule-based techniques <s> We address the problem of dealing with inconsistencies in fusion of big data sources using Resource Description Framework (RDF) and ontologies. We propose a scalable approach ensuring data quality for query answering over big RDF data in a distributed way on a Spark ecosystem. In so doing, the cleaning inconsistent big RDF data approach is built on the following steps (1) modeling consistency rules to detect the inconsistency triples even if it is implicitly hidden including inference and inconsistent rules (2) detecting inconsistency through rule evaluation based on Apache Spark framework to discover the minimally sub-set of inconsistent triples (3) cleaning the inconsistency through finding the best repair for consistent query answering. <s> BIB002 </s> Big Data Quality: A systematic literature review and future research directions <s> Rule-based techniques <s> The quality of data in wireless sensor networks has a significant impact on decision support, and data cleaning is an effective way to improve data quality. However, if the data cleaning strategies are not correctly designed, it might result in an unsatisfactory cleaning effect with increased system cleaning costs. Initially, data quality evaluation indicators and their measurement methods in wireless sensor networks were introduced. We then explored the impact of relationship between different indicators which are used in the quality assessment. Finally, data cleaning strategy for wireless sensor networks based on the relationship between data quality indicators was proposed by comparing and analyzing data cleaning schemes with different orders. The experimental results showed that the proposed data cleaning strategy can effectively improve data availability and have a better cleaning effect in wireless sensor networks for the same cleaning cost. <s> BIB003
Next, the explanation of the Rule-based methods is given, which has four papers. A rule-based methodology for detecting and cleaning inconsistency in triple data is presented in BIB002 . Inconsistency rules and logic are used to detect inconsistencies in knowledge bases. Then, the degree of inconsistency of each triple is determined by the definitions given in the paper, and finally, the data whose degree of inconsistency exceeds the desired threshold is eliminated. There is no evaluation section in this paper. In BIB001 , a big data preprocessing framework is provided. In this framework, the activity and the technique is selected first by the Data Quality Class Selection module. In activity, data integration or data cleaning is selected, and in the technique, one of the domain related, auto-discovery and user-defined can be selected. In domain related, predefined rules, specifically designed to address data quality problems frequently occurring in a specific domain, in auto-discovery, heuristics algorithms, and in the user-defined, some rules that are defined by users are applied to the data. Further, in the Data Quality module, quality dimensions and quality rules are determined. Then the activity is performed on a small volume of data, and if the evaluation results are approved, it runs on the entire data source. It is said that the framework is still not complete and is in development, so the evaluation of work has not been done. In BIB003 , a data cleaning scenario is proposed. Data volume, correctness, timeliness, and completeness are selected as quality dimensions. After examining the correlation between dimensions, the result is that there is a positive correlation between completeness and data volume, as well as when the received signal is continuous, there is also a positive correlation between timeliness and accuracy. Then six scenarios have been introduced to clean the data. By examining the correlation between dimensions, one of them is selected as the best scenario. In this scenario, firstly, the data completeness must be evaluated. Secondly, the data are analyzed in terms of the timeliness dimension, and finally, data correction must be verified. In the evaluation, the results of the dimensions correlation are specified, and the scenario is also compared with another scenario, which results show that the selected scenario has been able to clean the data better. A data cleaning framework for big batch data is presented in this dissertation  . This framework consists of identification and linking steps, recovering and association process, ranking system, and cleaning process. Identification is the process of matching objects from multiple data sources referring to the same entity. The linking is the process of linking together the identified objects, that is, the most closely associated objects identified from multiple sources that are related to the target object. The ranking system provides priority for data cleaning based on the severity of data issues. The metadata generator (MG) assists in identifying data items, types, and uniqueness for all types of datasets in the proposed framework. The MG generates both structural metadata and descriptive metadata when a new dataset is stored in the system. Structural metadata refers to the design and specification of data structures, like data item's name, data type, uniqueness, and the most commonly occurring patterns. Descriptive metadata refers to individual instances of application data or data content, like contents and context of data files. When data issues are identified and reported, the data quality administrator must determine the appropriate actions to be taken based on their severity. Data repairing is carried out based on obtained metadata. The results of the experiments have shown that the data quality has been improved using the proposed method.