1252625
Supporting self-adaptation in streaming data mining applications
There are many application classes where the users are flexible with respect to the output quality. At the same time, there are other constraints, such as the need for real-time or interactive response, which are more crucial. This paper presents and evaluates a runtime algorithm for supporting adaptive execution for such applications. The particular domain we target is distributed data mining on streaming data. This work has been done in the context of a middleware system called GATES (grid-based adaptive execution on streams) that we have been developing. The self-adaptation algorithm we present and evaluate in this paper has the following characteristics. First, it carefully evaluates the long-term load at each processing stage. It considers different possibilities for the load at a processing stage and its next stages, and decides if the value of an adaptation parameter needs to be modified, and if so, in which direction. To find the ideal new value of an adaptation parameter, it performs a binary search on the specified range of the parameter. To evaluate the self-adaptation algorithm in our middleware, we have implemented two streaming data mining applications. The main observations from our experiments are as follows. First, our algorithm is able to quickly converge to stable values of the adaptation parameter, for different data arrival rates, and independent of the specified initial value. Second, in a dynamic environment, the algorithm is able to adapt the processing rapidly. Finally, in both static and dynamic environments, the algorithm clearly outperforms the algorithm described in our earlier work and an obvious alternative, which is based on linear-updates.
6229211
Resource allocation in a middleware for streaming data
Increasingly, a number of applications rely on, or can potentially benefit from, analysis and monitoring of <i>data streams.</i> To support processing of streaming data in a grid environment, we have been developing a middleware system called GATES (Grid-based AdapTive Execution on Streams). Our target applications are those involving high volume data streams and requiring distributed processing of data arising from a distributed set of sources.
 This paper addresses the problem of resource allocation in the GATES system. Though resource discovery and resource allocation have been active topics in grid community, the pipelined processing and real-time constraint required by distributed streaming applications pose new challenges. We present a resource allocation algorithm that is based on minimal spanning trees. We evaluate the algorithm experimentally and demonstrate that it results in configurations that are very close to optimal, and significantly better than most other possible configurations.
27714693
Efficient Parallel Processing for K-Nearest-Neighbor Search in Spatial Databases
Even though the problem of k nearest neighbor (kNN) query is well-studied in serial environment, there is little prior work on parallel kNN search processing in parallel one. In this paper, we present the first Best-First based Parallel kNN (BFPkNN) query algorithm in a multi-disk setting, for efficient handling of kNN retrieval with arbitrary values of k by parallelization. The core of our method is to access more entries from multiple disks simultaneously and enable several effective pruning heuristics to discard non-qualifying entries. Extensive experiments with real and synthetic datasets confirm that BFPkNN significantly outperforms its competitors in both efficiency and scalability.
62256007
CapProbe: a simple and accurate capacity estimation technique
We present a new capacity estimation technique, called CapProbe. CapProbe combines delay as well as dispersion measurements of packet pairs to filter out samples distorted by cross-traffic. CapProbe algorithms include convergence tests and convergence speed-up techniques by varying probing parameters. Our study of CapProbe includes a probability analysis to determine the time it takes CapProbe to converge on the average. Through simulations and measurements, we found CapProbe to be quick and accurate across a wide range of traffic scenarios. We also compared CapProbe with two previous well-known techniques, pathchar and pathrate. We found CapProbe to be much more accurate than pathchar and similar in accuracy to pathrate, while providing faster estimation than both. Another advantage of CapProbe is its lower computation cost, since no statistical post processing of probing data is required.
24161285
Effects of Local-Lag Mechanism on Task Performance in a Desktop CVE System
Consistency maintenance is a kernel problem in Collaborative Virtual Environment (CVE) research. The approaches used in Networked Virtual Environments (e.g., DR algorithm) could not be used in CVEs, for they could not prevent short-term inconsistency. Therefore, local-lag mechanism has been proposed to eliminate short-term inconsistency in CVEs. Choosing a proper lag value is a key problem in local-lag mechanism. This paper studied the effects of lag value (0ms–900ms) on task performance in a desktop CVE system. Experimental results indicate that the effect of lag value on task performance is not linear. The effect could be separated into four segments by three dividing points: 150ms, 300ms and 600ms. Lag has no effect on task performance while ranging from 0ms to 150ms. From 150ms to 300ms, lag slightly affects task performance. Lag deteriorates task performance seriously while ranging from 300ms to 600ms. When lag is longer than 600ms, the task cannot be accomplished sometimes.
7800069
Detection of summative global predicates
In distributed programs, we usually keep some global predicates from being satisfied to make it easy to run the programs correctly. A common type of global predicates are: the total number of certain tokens in the whole distributed system is always the same or in specific ranges. In this paper, we call this summative predicates, classified into the following four: (1) at some global state of the system, N/spl ne/K, (2) NK (or N/spl ges/K), and (4) N=K, where N is the total number of tokens and K is a constant. This paper investigates the methods of detecting various summative global predicates. The first class of summative predicates are trivial to detect by simply checking each message. For the second class of summative predicates, B. Groselj (1993) and V.K. Garg (1995) solved the problem by reducing the problem to a maximum network flow problem. In this paper, we propose an elegant technique, called normalization, to allow the second and third classes of summative predicates to be solved by also reducing the problem to a maximum network flow problem. For the fourth class of summative predicates, we prove that it is an NP-complete problem.
7815226
Peak Load Forecasting Using the Self-organizing Map
This paper aims to study the short-term load forecasting of electricity by using an extended self-organizing map. We first adopt a traditional Kohonen self-organizing map (SOM) to learn time-series load data with weather information as parameters. Then, in order to improve the accuracy of the prediction, an extension of SOM algorithm based on error-correction learning rule is used, and the estimation of the peak load is achieved by averaging the output of all the neurons. Finally, as an implementation example, data of electricity demand from New York Independent System Operator (ISO) are used to verify the effectiveness of the learning and prediction for the proposed methods.