Time Series Data Cleaning: A Survey <s> C. KALMAN FILTER MODEL <s> Linearization errors inherent in the specification of an extended Kalman filter (EKF) can severely degrade its performance. This correspondence presents a new approach to the robust design of a discrete-time EKF by application of the robust linear design methods based on the H/sub /spl infin// norm minimization criterion. The results of simulations are presented to demonstrate an advantage for signal demodulation and nonlinear equalization applications. <s> BIB001 </s> Time Series Data Cleaning: A Survey <s> C. KALMAN FILTER MODEL <s> In this work, we are concerned with optimal estimation of clean speech from its noisy version based on a speech model we propose. We first propose a (single) speech model which satisfactorily describes voiced and unvoiced speech and silence (i.e., pauses between speech utterances), and also allows for exploitation of the long term characteristics of noise. We then reformulate the model equations so as to facilitate subsequent application of the well-established Kalman filter for computing the optimal estimate of the clean speech in the minimum-mean-square-error sense. Since the standard algorithm for Kalman-filtering involves multiplications of very large matrices and thus demands high computational cost, we devise a mathematically equivalent algorithm which is computationally much more efficient, by exploiting the sparsity of the matrices concerned. Next, we present the methods we use for estimating the model parameters and give a complete description of the enhancement process. Performance assessment based on spectrogram plots, objective measures and informal subjective listening tests all indicate that our method gives consistently good results. As far as signal-to-noise ratio is concerned, the improvements over existing methods can be as high as 4 dB. <s> BIB002 </s> Time Series Data Cleaning: A Survey <s> C. KALMAN FILTER MODEL <s> Although the extended Kalman filter is a widely used estimator for nonlinear systems, it has two drawbacks: linearization can produce unstable filters and it is hard to implement the derivation of the Jacobian matrices. This work presents a new method of predicting Mackey-Glass equation based on unscented Kalman filter. The principle of unscented transform is analyzed and the algorithm of UKF is discussed And then EKF and UKF methods are used to estimate the noisy chaotic time-series, and the estimation errors between two different algorithms are compared. Simulation results show this filter can predict chaotic time-series more effectively and accurately than extended Kalman filter. <s> BIB003 </s> Time Series Data Cleaning: A Survey <s> C. KALMAN FILTER MODEL <s> Consistent query answering is the problem of computing the answers from a database that are consistent with respect to certain integrity constraints that the database as a whole may fail to satisfy. Those answers are characterized as those that are invariant under minimal forms of restoring the consistency of the database. In this context, we study the problem of repairing databases by fixing integer numerical values at the attribute level with respect to denial and aggregation constraints. We introduce a quantitative definition of database fix, and investigate the complexity of several decision and optimization problems, including DFP, i.e. the existence of fixes within a given distance from the original instance, and CQA, i.e. deciding consistency of answers to aggregate conjunctive queries under different semantics. We provide sharp complexity bounds, identify relevant tractable cases; and introduce approximation algorithms for some of those that are intractable. More specifically, we obtain results like undecidability of existence of fixes for aggregation constraints; MAXSNP-hardness of DFP, but a good approximation algorithm for a relevant special case; and intractability but good approximation for CQA for aggregate queries for one database atom denials (plus built-ins). <s> BIB004 </s> Time Series Data Cleaning: A Survey <s> C. KALMAN FILTER MODEL <s> Abstract In Gardner [Gardner, E. S., Jr. (1985). Exponential smoothing: The state of the art. Journal of Forecasting 4, 1–28], I reviewed the research in exponential smoothing since the original work by Brown and Holt. This paper brings the state of the art up to date. The most important theoretical advance is the invention of a complete statistical rationale for exponential smoothing based on a new class of state-space models with a single source of error. The most important practical advance is the development of a robust method for smoothing damped multiplicative trends. We also have a new adaptive method for simple smoothing, the first such method to demonstrate credible improved forecast accuracy over fixed-parameter smoothing. Longstanding confusion in the literature about whether and how to renormalize seasonal indices in the Holt–Winters methods has finally been resolved. There has been significant work in forecasting for inventory control, including the development of new predictive distributions for total lead-time demand and several improved versions of Croston's method for forecasting intermittent time series. Regrettably, there has been little progress in the identification and selection of exponential smoothing methods. The research in this area is best described as inconclusive, and it is still difficult to beat the application of a damped trend to every time series. <s> BIB005 </s> Time Series Data Cleaning: A Survey <s> C. KALMAN FILTER MODEL <s> Nowadays, wireless sensor networks have been widely used in many monitoring applications. Due to the low quality of sensors and random effects of the environments, however, it is well known that the collected sensor data are noisy. Therefore, it is very critical to clean the sensor data before using them to answer queries or conduct data analysis. Popular data cleaning approaches, such as the moving average, cannot meet the requirements of both energy efficiency and quick response time in many sensor related applications. In this paper, we propose a hybrid sensor data cleaning approach with confidence. Specifically, we propose a smart weighted moving average (WMA) algorithm that collects confidence data from sensors and computes the weighted moving average. The rationale behind the WMA algorithm is to draw more samples for a particular value that is of great importance to the moving average, and provide higher confidence weight for this value, such that this important value can be quickly reflected in the moving average. Based on our extensive simulation results, we demonstrate that, compared to the simple moving average (SMA), our WMA approach can effectively clean data and offer quick response time. <s> BIB006 </s> Time Series Data Cleaning: A Survey <s> C. KALMAN FILTER MODEL <s> As key decisions are often made based on information contained in a database, it is important for the database to be as complete and correct as possible. For this reason, many data cleaning tools have been developed to automatically resolve inconsistencies in databases. However, data cleaning tools provide only best-effort results and usually cannot eradicate all errors that may exist in a database. Even more importantly, existing data cleaning tools do not typically address the problem of determining what information is missing from a database. To overcome the limitations of existing data cleaning techniques, we present QOCO, a novel query-oriented system for cleaning data with oracles. Under this framework, incorrect (resp. missing) tuples are removed from (added to) the result of a query through edits that are applied to the underlying database, where the edits are derived by interacting with domain experts which we model as oracle crowds. We show that the problem of determining minimal interactions with oracle crowds to derive database edits for removing (adding) incorrect (missing) tuples to the result of a query is NP-hard in general and present heuristic algorithms that interact with oracle crowds. Finally, we implement our algorithms in our prototype system QOCO and show that it is effective and efficient through a comprehensive suite of experiments. <s> BIB007
Kalman  proposes the Kalman filter theory, which can deal with time-varying systems, non-stationary signals, and multi-dimensional signals. Kalman filter creatively incorporates errors (predictive and measurement errors) into the calculation, the errors exist independently and are not affected by measured data. The Kalman model involves probability, random variable, Gaussian Distribution, and State-space Model, etc. Consider that the Kalman model involves too much content, no specific description is given here, and only a simple definition is given. First, we introduce a system of discrete control processes which can be described by a Linear Stochastic Difference equation as shown in equation BIB007 . Also, the measured values of the system are expressed as shown in equation BIB004 . In equation (7) and (8), x t is the system state value at time t, and v t is the control variable value for the system at time t. m and n are system parameters, and for multi-model systems, they are matrices, y t is the measured value at time t, r is the parameter of the measurement system, and for multi-measurement systems, r is a matrix, p(k) and q(k) represent the noises of the process and measurement, respectively, and they are assumed to be white Gaussian Noise. The extended Kalman filter is the most widely used estimation for a recursive nonlinear system because it simply linearizes the nonlinear system models. However, the extended Kalman filter has two drawbacks: linearization can produce unstable filters and it is hard to implement the derivation of the Jacobian matrices. Thereby, Ma and Teng BIB003 present a new method of predicting the Mackey-Glass equation based on the unscented Kalman filter to solve these problems. In the field of signal processing, there are many works [18] , BIB001 , BIB002 based on Kalman filtering, but these techniques have not been widely used in the field of time series cleaning. Gardner BIB005 propose a new model, which is based on the Kernel Kalman Filter, to perform various nonlinear time series processing. Zhuang et al. BIB006 use the Kalman filter model to predict sensor data and smoothed it with WMA.