6427165
Self-adaptation in a middleware for processing data streams
Many stream-based applications share a common set of characteristics, which makes grid-based and adaptive processing desirable or even a necessity. We view the problem of flexible and adaptive processing of distributed data streams as a grid computing problem. We have been developing a middleware for processing of distributed data streams. Our system is referred to as GATES (grid-based adaptive execution on streams). It is designed to use the existing grid standards and tools to the extent possible. Because of the real-time requirements, there is a need for adapting the processing in such a distributed environment, and achieving the best accuracy of the results within the real-time constraint. For this purpose, our system flexibly achieves the best accuracy that is possible while maintaining the real-time constraint on the analysis. We have developed and implemented a self-adaptation algorithm. This paper focuses on describing and evaluating this algorithm.
1252625
Supporting self-adaptation in streaming data mining applications
There are many application classes where the users are flexible with respect to the output quality. At the same time, there are other constraints, such as the need for real-time or interactive response, which are more crucial. This paper presents and evaluates a runtime algorithm for supporting adaptive execution for such applications. The particular domain we target is distributed data mining on streaming data. This work has been done in the context of a middleware system called GATES (grid-based adaptive execution on streams) that we have been developing. The self-adaptation algorithm we present and evaluate in this paper has the following characteristics. First, it carefully evaluates the long-term load at each processing stage. It considers different possibilities for the load at a processing stage and its next stages, and decides if the value of an adaptation parameter needs to be modified, and if so, in which direction. To find the ideal new value of an adaptation parameter, it performs a binary search on the specified range of the parameter. To evaluate the self-adaptation algorithm in our middleware, we have implemented two streaming data mining applications. The main observations from our experiments are as follows. First, our algorithm is able to quickly converge to stable values of the adaptation parameter, for different data arrival rates, and independent of the specified initial value. Second, in a dynamic environment, the algorithm is able to adapt the processing rapidly. Finally, in both static and dynamic environments, the algorithm clearly outperforms the algorithm described in our earlier work and an obvious alternative, which is based on linear-updates.
6229211
Resource allocation in a middleware for streaming data
Increasingly, a number of applications rely on, or can potentially benefit from, analysis and monitoring of <i>data streams.</i> To support processing of streaming data in a grid environment, we have been developing a middleware system called GATES (Grid-based AdapTive Execution on Streams). Our target applications are those involving high volume data streams and requiring distributed processing of data arising from a distributed set of sources.
 This paper addresses the problem of resource allocation in the GATES system. Though resource discovery and resource allocation have been active topics in grid community, the pipelined processing and real-time constraint required by distributed streaming applications pose new challenges. We present a resource allocation algorithm that is based on minimal spanning trees. We evaluate the algorithm experimentally and demonstrate that it results in configurations that are very close to optimal, and significantly better than most other possible configurations.
1912999
Improving wireless link throughput via interleaved FEC
Wireless communication is inherently vulnerable to errors from the dynamic wireless environment. Link layer packets discarded due to these errors impose a serious limitation on the maximum achievable throughput in the wireless channel. To enhance the overall throughput of wireless communication, it is necessary to have a link layer transmission scheme that is robust to the errors intrinsic to the wireless channel. To this end, we present interleaved-forward error correction (I-FEC), a clever link layer coding scheme that protects link layer data against random and busty errors. We examine the level of data protection provided by I-FEC against other popular schemes. We also simulate I-FEC in Bluetooth, and compare the TCP throughput result with Bluetooth's integrated FEC coding feature. We show that I-FEC consistently and significantly outperforms other link layer coding schemes by providing an impressive amount of protection against heavy channel burst errors.
8972258
Designing oscillators in synthetic gene networks based on multi-scale dynamics
Multistability, oscillations, and switching exist at various levels of biological processes and organizations and have been investigated on the basis of many theoretical models, such as circadian oscillations with the period protein (PER) and the timeless protein (TIM) in Drosophila, and multistable dynamics regulated by transcriptional factors. Considerable experimental evidence suggests that cellular processes are intrinsically rhythmic or periodic. Various periodic oscillations with different time scales ranging from less than a second to more than a year, which may allow for living organisms to adapt their behaviors to a periodically varying environment, have also been observed experimentally. On the other hand, in synthetic gene networks, both toggle switch and repressilator have been theoretically proposed and further confirmed by experiments. All of these works stress the importance of feedback regulation of transcriptional factors, which is a key in giving rise to oscillatory or multistable dynamical behaviors exhibited by biological genetic systems. In addition, it should be noted that many periodic behaviors do not simply oscillate smoothly; rather, they change rapidly or jump at certain states. In gene expression systems, many different time scales characterize the gene regulatory processes. For instance, the transcription and translation processes generally evolve on a time scale that is much slower than that of phosphorylation, dimerization or binding reactions of transcription factors. In genetic networks, the time scale for expression of some genes is much slower than that of others, depending on the length of the genes. We aim to design robust periodic oscillators in synthetic gene-protein systems by simple nonlinear models and to analyze the basic mechanism of limit cycles with jumping behaviors or relaxation oscillations by exploiting multiple time-scale properties [1, 2]. We show that periodic oscillations are mainly generated by nonlinear feedback loops in gene regulatory systems and the jumping dynamics caused by time scale differences among biochemical reactions. Moreover, effects of time delay are also examined. We show that time delay generally enlarges the stability region of oscillations, thereby making the oscillations more sustainable despite parameter changes or noise [1, 2]. The dynamics of the proposed models is robust in terms of stability and period length to the parameter perturbations or environment variations. Although we mainly analyze some specific models, the mechanisms identified in this work are likely to apply to a variety of genetic regulatory systems. These simple models may actually act as basic building block in synthetic gene-protein networks, such as genetic oscillators or switches because the dynamics is robust for parameter perturbations or environment variations. Several examples are also provided to demonstrate implementation of synthetic oscillators by using genes of the /spl lambda/ phage bacteria.
206977606
Priority Based Overlay Multicast for Distributed Interactive Applications
Distributed Interactive Applications (DIAs) is different with other network-based applications, since the data should be delivered to the receiver node according to its priority. Among the existing overlay multicast protocols the "priority-based directed minimum spanning tree" (PST) is designed for DIAs. But in PST the available bandwidth (avail-bw) is used inefficiently and the system becomes unstable when its scale increases. This paper proposed a novel overlay multicast protocol named Priority Based Overlay Multicast (POM), which uses the entities’ priority, links’ delay and node’s avail-bw to build multicast tree. By using POM the nodes with high priority receive the data in short delay, while the nodes with low priority tolerate a long delay in order to balance the available bandwidth usage among the nodes. The simulation results show that POM uses the avail-bw efficiently and keeps the system stable when its scale increases. Keywords: DIAs; Priority; Available bandwidth.
44509988
A threshold criterion, auto-detection and its use in MST-based clustering
Clustering is to group data points into homogenous clusters so that data points within the same cluster are more similar than data points belonging to different clusters. There are many effective clustering algorithms for discovering arbitrary shaped clusters, but one common problem of many algorithms is the difficulty for users to decide appropriate parameters for these algorithms. To reduce the dependence of clustering performance on parameters, this paper proposes a threshold criterion for the single linkage cluster analysis and incorporates it into the Minimum Spanning Tree (MST) based clustering method. Since the threshold can be automatically decided according to the underlying data distributions, arbitrary shaped clusters can be discovered with little human intervention. The experimental results on spatial data are very encouraging.
8213932
End-to-End Asymmetric Link Capacity Estimation
Knowledge of link capacity is important for network design, management, and utilization. With the increasing popularity of asymmetric link technologies (such as DSL, 1xRTT, and satellite links), it is desirable to have a capacity estimation technique, which can simultaneously measure forward and backward direction link capacities on an Internet path. Moreover, this estimation must often be “sender only”, because of receiver limitations or lack of standards. In this study, we propose a simple, fast and accurate technique, called AsymProbe, to estimate asymmetric link capacities. AsymProbe is a “sender only”, round trip procedure. It achieves asymmetric link capacity estimation by strategically altering the ratio of probe and acknowledgement packet sizes. Using simulation and testbed experiments, we validate AsymProbe with a variety of network configurations. The results show that AsymProbe can correctly estimate the asymmetric link capacities as long as an appropriate packet size ratio can be employed.