Energy big data: A survey <s> B. DATA QUALITY <s> The performance of a pattern recognition system is dependent on, among other things, an appropriate data-preprocessing technique, In this paper, we describe a method to evaluate the performance of a variety of these techniques for the problem of odour classification using an array of gas sensors, also referred to as an electronic nose. Four experimental odour databases with different complexities are used to score the data-preprocessing techniques. The performance measure used is the cross-validation estimate of the classification rate of a K nearest neighbor voting rule operating on Fisher's linear discriminant projection subspace. <s> BIB001 </s> Energy big data: A survey <s> B. DATA QUALITY <s> We study the problem of clustering data objects whose locations are uncertain. A data object is represented by an uncertainty region over which a probability density function (pdf) is defined. One method to cluster uncertain objects of this sort is to apply the UK-means algorithm, which is based on the traditional K-means algorithm. In UK-means, an object is assigned to the cluster whose representative has the smallest expected distance to the object. For arbitrary pdf, calculating the expected distance between an object and a cluster representative requires expensive integration computation. We study various pruning methods to avoid such expensive expected distance calculation. <s> BIB002
Due to the multiple origins of big data in the complicated power system, the smart grid database always contains incomplete, inconsistent and uncorrected data. Therefore, a series of data preprocessing technologies are necessary to improve the data quality, especially after uncertain data mining and imprecise querying BIB002 . Data preprocessing contains data manipulation, compression and normalization of the older data into improved format BIB001 . Data manipulation refers to transformation, which is used to reduce the impact from data fault sensing and provide more reliable energy data. The compression operation is used for massive data measurements to be reduced into a reasonable extent. Meanwhile, the normalization technology is used to regulate the collected data by minimizing its noise, inconsistencies and incompleteness of data. These three operations are main components of data preprocessing but the order is not unique. Furthermore, different regulations on big data cause conflicts between collaborators and lead to inconvenience in smart grid. Therefore, a unified and complete system standard needs to be set up in the future.