In power systems, load curve data is one of the most important datasets that are collected and retained by utilities. The quality of load curve data, however, is hard to guarantee since the data is subject to communication losses, meter malfunctions, and many other impacts. In this paper, a new approach to analyzing load curve data is presented. The method adopts a new view, termed , on the load curve data by analyzing the periodic patterns in the data and re-organizing the data for ease of analysis. Furthermore, we introduce algorithms to build the virtual portrait load curve data, and demonstrate its application on load curve data cleansing. Compared to existing regression-based methods, our method is much faster and more accurate for both small-scale and large-scale real-world datasets.
Some of the above methods, especially the regression-based methods, have also been used for load forecasting. Nevertheless, load forecasting and load data cleansing are different applications, and their purposes are different. For load forecasting, as mentioned in @cite_14 , all historical data are trusted and used to forecast the load at a future point in time. For load cleansing, historical records are used to detect corrupted data at a historical point in time, and appropriate values may be needed to replace the corrupted data. Therefore, load cleansing and load forecasting belong to different phases of load analysis: load cleansing comes before load forecasting and provides accurate load information for the latter.
@cite_14
2097490816 2407858842 2241020437 2109440766
Solid state drives (SSDs) update data by writing a new copy, rather than overwriting old data, causing prior copies of the same data to be invalidated. These writes are performed in units of pages, while space is reclaimed in units of multi-page erase blocks, necessitating copying of any remaining valid pages in the block before reclamation. The efficiency of this cleaning process greatly affects performance under random workloads; in particular, in SSDs the write bottleneck is typically internal media throughput, and write amplification due to additional internal copying directly reduces application throughput. We present the first precise closed-form solution for write amplification under greedy cleaning for uniformly distributed random traffic, and validate its accuracy via simulation. In addition we also present the first models which predict performance degradation for both LRU cleaning and greedy cleaning under simple non-uniform traffic conditions; simulation results show the first model to be exact and the second to be accurate within 2 . We extend the LRU model to arbitrary combinations of random traffic, and demonstrate its use in predicting cleaning performance for real-world workloads. The utility of these analytic models lies in their amenability to optimization techniques not feasible in simulation. We examine the strategy of separating "hot" and "cold" data, showing that for our traffic model, such separation eliminates any loss in performance due to non-uniform traffic. We show how a system which separates hot and cold data may shift free space from cold to hot data in order to achieve improved performance, and how numeric methods may be used with our model to find the optimum operating point, which approaches a write amplification of 1.0 for increasingly skewed traffic. We examine online methods for achieving this optimal operating point, and show that a control strategy based on our model achieves near-optimal performance for a number of real-world block traces. Heavy precipitation for several days and weeks always leads to some extreme nature disasters. Long-lead term precipitation forecasting plays an important role on the prevision of such calamities. Most works focus on the generation of training labels with allocation of the proper corresponding spatio-temporal information. In this paper, we will provide a different path by performing regression analysis using the precipitation amounts at particular locations. This method is called Hierarchical Clustering based Bayesian Structural Vector Autoregression (HC-BSVAR). The approach for HC-BSVAR is divided into two steps. First, we apply a hierarchical clustering algorithm to identify the Elite locations and then transfer the 3-dimensional data space into a new traditional 2-dimensional data space. Every column of the new data frame is a hydro-meteorological feature of the original data and each row represents a time point (day) in the original space. Secondly, an economic-based multivariate time series model called Bayesian-based Structural Vector Autoregression (BSVAR) is exploited to perform the final prediction result. The prediction quality will be vary for different cut of tree structure which generated by hierarchical clustering. The coefficient for determination of each location by each level of cut is applied to quantize the quality of prediction. The relationship between the cut level of clustering geographic locations and the regression model performance are also discussed, based on the result of prediction quality. We consider MapReduce workloads that are produced by analytics applications. In contrast to ad hoc query workloads, analytics applications are comprised of fixed data flows that are run over newly arriving data sets or on different portions of an existing data set. Examples of such workloads include document analysis indexing, social media analytics, and ETL (Extract Transform Load). Motivated by these workloads, we propose a technique that predicts the runtime performance for a fixed set of queries running over varying input data sets. Our prediction technique splits each query into several segments where each segment’s performance is estimated using machine learning models. These per-segment estimates are plugged into a global analytical model to predict the overall query runtime. Our approach uses minimal statistics about the input data sets (e.g., tuple size, cardinality), which are complemented with historical information about prior query executions (e.g., execution time). We analyze the accuracy of predictions for several segment granularities on both standard analytical benchmarks such as TPC-DS [17], and on several real workloads. We obtain less than 25 prediction errors for 90 of predictions. In this paper we examine the problem of balancing load in a large-scale distributed system when information about server loads may be stale. It is well known that sending each request to the machine with the apparent lowest load can behave badly in such systems, yet this technique is common in practice. Other systems use round-robin or random selection algorithms that entirely ignore load information or that only use a small subset of the load information. Rather than risk extremely bad performance on one hand or ignore the chance to use load information to improve performance on the other, we develop strategies that interpret load information based on its age. Through simulation, we examine several simple algorithms that use such load interpretation strategies under a range of workloads. Our experiments suggest that by properly interpreting load information, systems can (1) match the performance of the most aggressive algorithms when load information is fresh relative to the job arrival rate, (2) outperform the best of the other algorithms we examine by as much as 60 when information is moderately old, (3) significantly outperform random load distribution when information is older still, and (4) avoid pathological behavior even when information is extremely old.