Time Series Data Cleaning: A Survey <s> C. BINOMIAL SAMPLING <s> For many KDD applications, such as detecting criminal activities in E-commerce, finding the rare instances or the outliers, can be more interesting than finding the common patterns. Existing work in outlier detection regards being an outlier as a binary property. In this paper, we contend that for many scenarios, it is more meaningful to assign to each object a degree of being an outlier. This degree is called the local outlier factor (LOF) of an object. It is local in that the degree depends on how isolated the object is with respect to the surrounding neighborhood. We give a detailed formal analysis showing that LOF enjoys many desirable properties. Using real-world datasets, we demonstrate that LOF can be used to find outliers which appear to be meaningful, but can otherwise not be identified with existing approaches. Finally, a careful performance evaluation of our algorithm confirms we show that our approach of finding local outliers can be practical. <s> BIB001 </s> Time Series Data Cleaning: A Survey <s> C. BINOMIAL SAMPLING <s> To compensate for the inherent unreliability of RFID data streams, most RFID middleware systems employ a "smoothing filter", a sliding-window aggregate that interpolates for lost readings. In this paper, we propose SMURF, the first declarative, adaptive smoothing filter for RFID data cleaning. SMURF models the unreliability of RFID readings by viewing RFID streams as a statistical sample of tags in the physical world, and exploits techniques grounded in sampling theory to drive its cleaning processes. Through the use of tools such as binomial sampling and π-estimators, SMURF continuously adapts the smoothing window size in a principled manner to provide accurate RFID data to applications. <s> BIB002 </s> Time Series Data Cleaning: A Survey <s> C. BINOMIAL SAMPLING <s> Radio frequency identification (RFID) technology has seen increasing adoption rates in applications that range from supply chain management, asset tracking, Medical/Health Care applications, People tracking, Manufacturing, Retail, Warehouses, and Livestock Timing. This technology is used in many applications for data collection. The data captured by RFID readers are usually of low quality and may contain many anomalies. Data quality has become increasingly important to many organizations. This is especially true in the Medical/health care field because minute errors in it can cost heavy financial and personal losses. In order to provide reliable data to RFID application it is necessary to clean the collected data. SMURF is a declarative and adaptive smoothing cleaning technique for unreliable RFID data. However it does not work well when tag moves rapidly in and out of reader's communication range. The errors need to be cleansed in an effective manner before they are subjected to warehousing. Factors such as inter tag distance, tag-antenna distance, number of tags in the read range of antenna, reader communication range, velocity of tag movement affect the data cleaning result. Our proposed algorithm considers these factors and also the missing tag information, tags that are mistakenly read as present dynamically in determination of the size of slide window. Simulation shows our cleansing approach deals with RFID data more accurately and efficiently. Thus with the aid of the planned data cleaning technique we can bring down the health care costs, optimize business processes, streamline patient identification processes and improve patient safety. <s> BIB003 </s> Time Series Data Cleaning: A Survey <s> C. BINOMIAL SAMPLING <s> RFID technology is widely used in the Internet of Things (IoT) environment for object tracking. With the expansion of its application areas, the demand for reliability of business data is increasingly important. In order to fulfil the needs of upper-level applications, data cleaning is essential and directly affects the correctness and completeness of the business data, so it needs to filter and handle RFID data. The traditional statistical smoothing for unreliable RFID data (SMURF) algorithm is only aimed at constant speed data flow during the process of data cleaning. In this paper, we overcome the shortage of SMURF algorithm, and an improved SMURF scheme in two aspects is proposed. The first one is based on dynamic tags, and the second one considers the influence of data redundancy. The experiments verify that the improved scheme is reasonable in dynamic settings of sliding window, and the accuracy is improved as well. <s> BIB004
Jeffery et al. BIB002 propose an adaptive method for cleaning RFID data, which exploits techniques based on sampling and smoothing theory to improve the quality of RFID data. Tag transition detection: tag transition detection refers to the fact that when the position of the tag changes, the cleaning result should reflect the fact that the tag leaves. Let's introduce a few RFID-related concepts. (1) Interrogation cycles: the reader's question-and-answer process for the tag is the basic unit of the reader's detection tag. (2) Reader read cycle (epoch, 0.2 sec -0.25 sec): a collection of multiple interrogation cycles. Based on the above concept, the following definition: W i is smooth window of tag i and is composed of ω i epoch, S i is the window that tag i is actually detected in the W i window, Count t indicates the number of inquiry cycles of t, and R is the corresponding number of t epoch tag i. For a given time window, suppose the probability that the tag i may be read in each epoch is p i = R Count t , and the Statistical Soothing for Unreliable RFID Data (SMURF) BIB002 algorithm treats each epoch's reading of the tag as a Bernoulli experiment with probability p i . Therefore, p i conforms to the binomial distribution B(ω i , p i ). p i,avg is the average read rate in S i . Using the model based on the Bernoulli experiment to observe the tag i, if the average reading rate of the tags in ω i epoch is (1 − p i,avg ) ω i . To ensure the dynamic nature of the tag the size of the sliding window W i needs to be satisfied as shown in equation BIB001 . The SMURF algorithm first sets the initial window size to 1, and then dynamically adjusts the window length based on the actual situation of the read. If the current window meets the integrity requirement BIB002 , the SMURF algorithm will detect the status of the tag. When the detection result indicates that the tag status changes, SMURF will adjust the current window length to 1/2 of the original window to react to the tag's transition. If the calculated window size that satisfies the integrity constraint is greater than the current window size, the algorithm linearly increases the current window size by 2 steps and outputs the point data in the current window. If it is detected that the label does not move, the algorithm outputs the current window midpoint as the output point, and then continues to slide an epoch for the next processing. SMURF algorithm is widely used to clean RFID data, and many studies BIB003 , BIB004 improve it. Leema et al. BIB003 study the effect of tag movement speed on data removal results and Xu et al. BIB004 consider the impact of data redundancy on setting up sliding windows.