Time Series Data Cleaning: A Survey <s> B. FUTURE DIRECTIONS <s> Outliers, level shifts, and variance changes are commonplace in applied time series analysis. However, their existence is often ignored and their impact is overlooked, for the lack of simple and useful methods to detect and handle those extraordinary events. The problem of detecting outliers, level shifts, and variance changes in a univariate time series is considered. The methods employed are extremely simple yet useful. Only the least squares techniques and residual variance ratios are used. The effectiveness of these simple methods is demonstrated by analysing three real data sets. <s> BIB001 </s> Time Series Data Cleaning: A Survey <s> B. FUTURE DIRECTIONS <s> Stream data are often dirty, for example, owing to unreliable sensor reading, or erroneous extraction of stock prices. Most stream data cleaning approaches employ a smoothing filter, which may seriously alter the data without preserving the original information. We argue that the cleaning should avoid changing those originally correct/clean data, a.k.a. the minimum change principle in data cleaning. To capture the knowledge about what is clean, we consider the (widely existing) constraints on the speed of data changes, such as fuel consumption per hour, or daily limit of stock prices. Guided by these semantic constraints, in this paper, we propose SCREEN, the first constraint-based approach for cleaning stream data. It is notable that existing data repair techniques clean (a sequence of) data as a whole and fail to support stream computation. To this end, we have to relax the global optimum over the entire sequence to the local optimum in a window. Rather than the commonly observed NP-hardness of general data repairing problems, our major contributions include: (1) polynomial time algorithm for global optimum, (2) linear time algorithm towards local optimum under an efficient Median Principle,(3) support on out-of-order arrivals of data points, and(4) adaptive window size for balancing repair accuracy and efficiency. Experiments on real datasets demonstrate that SCREEN can show significantly higher repair accuracy than the existing approaches such as smoothing. <s> BIB002 </s> Time Series Data Cleaning: A Survey <s> B. FUTURE DIRECTIONS <s> Errors are prevalent in data sequences, such as GPS trajectories or sensor readings. Existing methods on cleaning sequential data employ a constraint on value changing speeds and perform constraint-based repairing. While such speed constraints are effective in identifying large spike errors, the small errors that do not significantly deviate from the truth and indeed satisfy the speed constraints can hardly be identified and repaired. To handle such small errors, in this paper, we propose a statistical based cleaning method. Rather than declaring a broad constraint of max/min speeds, we model the probability distribution of speed changes. The repairing problem is thus to maximize the likelihood of the sequence w.r.t. the probability of speed changes. We formalize the likelihood-based cleaning problem, show its NP-hardness, devise exact algorithms, and propose several approximate/heuristic methods to trade off effectiveness for efficiency. Experiments on real data sets (in various applications) demonstrate the superiority of our proposal. <s> BIB003 </s> Time Series Data Cleaning: A Survey <s> B. FUTURE DIRECTIONS <s> Errors are prevalent in time series data, such as GPS trajectories or sensor readings. Existing methods focus more on anomaly detection but not on repairing the detected anomalies. By simply filtering out the dirty data via anomaly detection, applications could still be unreliable over the incomplete time series. Instead of simply discarding anomalies, we propose to (iteratively) repair them in time series data, by creatively bonding the beauty of temporal nature in anomaly detection with the widely considered minimum change principle in data repairing. Our major contributions include: (1) a novel framework of iterative minimum repairing (IMR) over time series data, (2) explicit analysis on convergence of the proposed iterative minimum repairing, and (3) efficient estimation of parameters in each iteration. Remarkably, with incremental computation, we reduce the complexity of parameter estimation from O(n) to O(1). Experiments on real datasets demonstrate the superiority of our proposal compared to the state-of-the-art approaches. In particular, we show that (the proposed) repairing indeed improves the time series classification application. <s> BIB004 </s> Time Series Data Cleaning: A Survey <s> B. FUTURE DIRECTIONS <s> Smart home presents a challenge in control and monitoring of its wireless sensors networks (WSN) and the internet of things (IoT) devices which form it. The current IoT architectures are centralized, complex, with poor security in its communications and with upstream communication channels mainly. As a result, there are problems with data reliability. These problems include data missing, malicious data inserted, communications network overload, and overload of computing power at the central node. In this paper a new architecture is presented. This architecture based in blockchain introduce the edge computing layer and a new algorithm to improve data quality and false data detection. <s> BIB005
As mentioned above, data is an intangible asset and advanced technology helps to fully exploit the potential value of data. Thereby, time series data cleaning methods provide very important technical support for the discovery of these values in processing time series error data. Next we list some advice of future directions based on  . The error type illustrates handbook of time series data. At present, data scientists have a very detailed analysis of the errors in the traditional relational database. However, there is still much work to be further studied in the analysis of time series data error types. For instance, this paper roughly divides the types of time series errors into three types, namely single point big errors, single point small errors and continuous errors. In fact, in continuous errors, there are also a lot of meticulous types of errors, such as additive errors and the innovational errors BIB001 . How to systematically analyze these error types and form time series data error type illustrated handbook is very important. The clear error type helps to develop targeted cleaning algorithms to solve the problem of ''GIGO (Garbage in, garbage out.)'' that exists in the current field. The design of time series data cleaning algorithm. Each chapter of this paper reviews some time series error cleaning algorithms, but further optimizations are possible. The existing methods are mostly for a single-dimensional time series (even the GPS data exists two dimensions' information), but each dimension is cleaned separately during cleaning BIB002 , BIB003 , BIB004 . To further improve the practicability of the algorithm, it is imperative to consider the cleaning of multidimensional time series. Besides, with the development of machine learning technology, more technical learning techniques should be considered for data cleaning algorithms, which may lead to better cleaning results because of the mathematical support behind them. The implementation of time series cleaning tool. At present, the mainstream data cleaning tools in the industry are still aimed at relational databases, and these tools are not ideal for processing time series data. As time series data cleaning problems become more serious, how to use the fast-developing distributed technology, high performance computing technology and stream processing technology to implement time series cleaning tools (including research tools and commercial tools) and apply them to real-world scenarios such as industry is also the key work of the next stage. The algorithm design of time series anomaly detection. In real-world scenarios, efficient anomaly detection algorithms play an irreplaceable role in time series repair. It is difficult to judge the difference between the error value and the true value, so it is necessary to specifically design a time series anomaly detection algorithm that can be applied to an industrial scene. It is worth noting that more research is needed on how to perform anomaly detection, cleaning, and analysis in the case of weak domain knowledge or less labeled data. Design of data cleaning algorithms for specific application scenarios. With the application of various technologies in the industry, the application scenarios are becoming more and more clear. The requirements for cleaning algorithms in different application scenarios have different focuses. For instance, the data stored in the Blockchain network BIB005 are generally structured data, with the development of Blockchain technology, the design of data cleaning algorithms on Blockchain networks is also particularly important.