3433910
Residual-Based Measurement of Peer and Link Lifetimes in Gnutella Networks
Existing methods of measuring lifetimes in P2P systems usually rely on the so-called create-based method (CBM), which divides a given observation window into two halves and samples users "created" in the first half every Delta time units until they die or the observation period ends. Despite its frequent use, this approach has no rigorous accuracy or overhead analysis in the literature. To shed more light on its performance, we flrst derive a model for CBM and show that small window size or large Delta may lead to highly inaccurate lifetime distributions. We then show that create-based sampling exhibits an inherent tradeoff between overhead and accuracy, which does not allow any fundamental improvement to the method. Instead, we propose a completely different approach for sampling user dynamics that keeps track of only residual lifetimes of peers and uses a simple renewal-process model to recover the actual lifetimes from the observed residuals. Our analysis indicates that for reasonably large systems, the proposed method can reduce bandwidth consumption by several orders of magnitude compared to prior approaches while simultaneously achieving higher accuracy. We finish the paper by implementing a two-tier Gnutella network crawler equipped with the proposed sampling method and obtain the distribution of ultrapeer lifetimes in a network of 6.4 million users and 60 million links. Our experimental results show that ultrapeer lifetimes are Pareto with shape a alpha ap 1.1; however, link lifetimes exhibit much lighter tails with alpha ap 1.9.
1958263
On zone-balancing of peer-to-peer networks: analysis of random node join
Balancing peer-to-peer graphs, including zone-size distributions, has recently become an important topic of peer-to-peer (P2P) research [1], [2], [6], [19], [31], [36]. To bring analytical understanding into the various peer-join mechanisms, we study how zone-balancing decisions made during the initial sampling of the peer space affect the resulting zone sizes and derive several asymptotic results for the maximum and minimum zone sizes that hold with high probability.
322407
IRLbot: Scaling to 6 billion pages and beyond
This article shares our experience in designing a Web crawler that can download billions of pages using a single-server implementation and models its performance. We first show that current crawling algorithms cannot effectively cope with the sheer volume of URLs generated in large crawls, highly branching spam, legitimate multimillion-page blog sites, and infinite loops created by server-side scripts. We then offer a set of techniques for dealing with these issues and test their performance in an implementation we call IRLbot. In our recent experiment that lasted 41 days, IRLbot running on a single server successfully crawled 6.3 billion valid HTML pages (7.6 billion connection requests) and sustained an average download rate of 319 mb/s (1,789 pages/s). Unlike our prior experiments with algorithms proposed in related work, this version of IRLbot did not experience any bottlenecks and successfully handled content from over 117 million hosts, parsed out 394 billion links, and discovered a subset of the Web graph with 41 billion unique nodes.
1202188
Load-Balancing Performance of Consistent Hashing: Asymptotic Analysis of Random Node Join
Balancing of structured peer-to-peer graphs, including their zone sizes, has recently become an important topic of distributed hash table (DHT) research. To bring analytical understanding into the various peer-join mechanisms based on consistent hashing, we study how zone-balancing decisions made during the initial sampling of the peer space affect the resulting zone sizes and derive several asymptotic bounds for the maximum and minimum zone sizes that hold with high probability. Several of our results contradict those of prior work and shed new light on the theoretical performance limitations of consistent hashing. We use simulations to verify our models and compare the performance of the various methods using the example of recently proposed de Bruijn DHTs.
3443754
Residual-Based Estimation of Peer and Link Lifetimes in P2P Networks
Existing methods of measuring lifetimes in P2P systems usually rely on the so-called Create-Based Method (CBM), which divides a given observation window into two halves and samples users ldquocreatedrdquo in the first half every Delta time units until they die or the observation period ends. Despite its frequent use, this approach has no rigorous accuracy or overhead analysis in the literature. To shed more light on its performance, we first derive a model for CBM and show that small window size or large Delta may lead to highly inaccurate lifetime distributions. We then show that create-based sampling exhibits an inherent tradeoff between overhead and accuracy, which does not allow any fundamental improvement to the method. Instead, we propose a completely different approach for sampling user dynamics that keeps track of only residual lifetimes of peers and uses a simple renewal-process model to recover the actual lifetimes from the observed residuals. Our analysis indicates that for reasonably large systems, the proposed method can reduce bandwidth consumption by several orders of magnitude compared to prior approaches while simultaneously achieving higher accuracy. We finish the paper by implementing a two-tier Gnutella network crawler equipped with the proposed sampling method and obtain the distribution of ultrapeer lifetimes in a network of 6.4 million users and 60 million links. Our experimental results show that ultrapeer lifetimes are Pareto with shape alpha ap 1.1; however, link lifetimes exhibit much lighter tails with alpha ap 1.8.
47232248
Graph-theoretic analysis of structured peer-to-peer systems: routing distances and fault resilience
This paper examines graph-theoretic properties of existing peer-to-peer networks and proposes a new infrastructure based on optimal-diameter de Bruijn graphs. Since generalized de Bruijn graphs exhibit very short average distances and high resilience to node failure, they are well suited for distributed hash tables (DHTs). Using the example of Chord, CAN, and de Bruijn, we study the routing performance, graph expansion, clustering properties, and bisection width of each graph. Having confirmed that de Bruijn graphs offer the best diameter and highest connectivity among the existing peer-to-peer structures, we offer a very simple incremental building process that preserves optimal properties of de Bruijn graphs under uniform user joins/departures. We call the combined peer-to-peer architecture optimal diameter routing infrastructure.
3479727
p-Percent Coverage Schedule in Wireless Sensor Networks
We investigate the p-percent coverage problem in this paper and propose two algorithms to prolong network lifetime based on the fact that for some applications full coverage is not necessary and different subareas of the monitored area may have different coverage requirements. The first algorithm, CPCA, is a centralized algorithm which selects the least number of nodes to monitor p-percent of the monitored area. The second algorithm, DPCP, is a distributed algorithm which can determine a set of nodes in a distributed manner to cover p-percent of the monitored area. Both of the algorithms guarantee network connectivity. The simulation results show that our algorithms can remarkably prolong network lifetime, have less than 5% unrequired coverage for large networks and employ nodes fairly for most cases.
20162487
An efficient and scalable pairwise key pre-distribution scheme for sensor networks using deployment knowledge
Pairwise key establishment is a fundamental security service for sensor networks. However, establishing pairwise key in sensor networks is a challenging problem, particularly due to the resource constraints on sensor nodes and the threat of node compromises. On the other hand, adding new nodes to a sensor network is a fundamental requirement for their continuous operation over time, too. We analyze the weaknesses of security due to node capture when adding sensor nodes using key pre-distribution schemes with ''fixed'' key pools. In this paper, we propose a new approach, which separates the nodes into groups, the nodes in a group communicate with each other using pairwise keys pre-distributed, the communications between any two neighbor groups are accomplished also through pairwise keys, which is computed based on the pre-distributed Hash chain. We show that the performance (e.g. continuous connectivity, continuous network resilience against node capture and memory usage) of sensor networks can be substantially improved by using our scheme. The scheme and its detailed performance evaluation are presented.
37286032
A Random Key Management Scheme for Wireless Sensor Networks
The establishment of shared keys between communicating neighbor nodes in wireless sensor networks is a challenge due to resource-constrained sensor networks. Several key pre-distribution schemes have been proposed in literatures to establish pairwise keys between sensor nodes. However, many of them are either too complicated to fit for wireless sensor networks or insecure for some common aggressions. In this paper, we propose a random key management scheme based on random key pre-distribution for wireless sensor networks. To achieve better performance and security, the proposed scheme employs two different phases to establish enhanced pairwise keys between neighboring nodes. Compared with other existing random key pre-distribution schemes, our scheme has better resilience against node capture and also performs better in terms of network connectivity and scalability with appropriate memory, computation and communication overheads.
17025259
A level-set based variational method for design and optimization of heterogeneous objects
Abstract A heterogeneous object is referred to as a solid object made of different constituent materials. The object is of a finite collection of regions of a set of prescribed material classes of continuously varying material properties. These properties have a discontinuous change across the interface of the material regions. In this paper, we propose a level-set based variational approach for the design of this class of heterogeneous objects. Central to the approach is a variational framework for a well-posed formulation of the design problem. In particular, we adapt the Mumford–Shah model which specifies that any point of the object belongs to either of two types: inside a material region of a well-defined gradient or on the boundary edges and surfaces of discontinuities. Furthermore, the set of discontinuities is represented implicitly, using a multi-phase level set model. This level-set based variational approach yields a computational system of coupled geometric evolution and diffusion partial differential equations. Promising features of the proposed method include strong regularity in the problem formulation and inherent capabilities of geometric and material modeling, yielding a common framework for optimization of the heterogeneous objects that incorporates dimension, shape, topology, and material properties. The proposed method is illustrated with several 2D examples of optimal design of multi-material structures and materials.
25804073
Translational integrity and continuity: Personalized biomedical data integration
Translational research data are generated in multiple research domains from the bedside to experimental laboratories. These data are typically stored in heterogeneous databases, held by segregated research domains, and described with inconsistent terminologies. Such inconsistency and fragmentation of data significantly impedes the efficiency of tracking and analyzing human-centered records. To address this problem, we have developed a data repository and management system named TraM (http://tram.uchicago.edu), based on a domain ontology integrated entity relationship model. The TraM system has the flexibility to recruit dynamically evolving domain concepts and the ability to support data integration for a broad range of translational research. The web-based application interfaces of TraM allow curators to improve data quality and provide robust and user-friendly cross-domain query functions. In its current stage, TraM relies on a semi-automated mechanism to standardize and restructure source data for data integration and thus does not support real-time data application.