1284213
An algebraic approach to practical and scalable overlay network monitoring
Overlay network monitoring enables distributed Internet applications to detect and recover from path outages and periods of degraded performance within seconds. For an overlay network with n end hosts, existing systems either require O(n2) measurements, and thus lack scalability, or can only estimate the latency but not congestion or failures. Our earlier extended abstract [1] briefly proposes an algebraic approach that selectively monitors k linearly independent paths that can fully describe all the O(n2) paths. The loss rates and latency of these k paths can be used to estimate the loss rates and latency of all other paths. Our scheme only assumes knowledge of the underlying IP topology, with links dynamically varying between lossy and normal.In this paper, we improve, implement and extensively evaluate such a monitoring system. We further make the following contributions: i) scalability analysis indicating that for reasonably large n (e.g., 100), the growth of k is bounded as O(n log n), ii) efficient adaptation algorithms for topology changes, such as the addition or removal of end hosts and routing changes, iii) measurement load balancing schemes, and iv) topology measurement error handling. Both simulation and Internet experiments demonstrate we obtain highly accurate path loss rate estimation while adapting to topology changes within seconds and handling topology errors.
10399949
Quantifying Network Denial of Service: A Location Service Case Study
Network Denial of Service (DoS) attacks are increasing in frequency, severity and sophistication, making it desirable to measure the resilience of systems to DoS attacks. In this paper, we propose a simulation-based methodology and apply it to attacks on object location services such as DNS. Our results allow us to contrast the DoS resilience of three distinct architectures for object location.
10897641
Reversible sketches for efficient and accurate change detection over network data streams
Traffic anomalies such as failures and attacks are increasing in frequency and severity, and thus identifying them rapidly and accurately is critical for large network operators. The detection typically treats the traffic as a collection of flows and looks for heavy changes in traffic patterns (<i>e.g.</i>, volume, number of connections). However, as link speeds and the number of flows increase, keeping per-flow state is not scalable. The recently proposed sketch-based schemes [14] are among the very few that can detect heavy changes and anomalies over massive data streams at network traffic speeds. However, sketches do not preserve the key (<i>e.g.</i>, source IP address) of the flows. Hence, even if anomalies are detected, it is difficult to infer the culprit flows, making it a big practical hurdle for online deployment. Meanwhile, the number of keys is too large to record.
 To address this challenge, we propose efficient <i>reversible hashing</i> algorithms to infer the keys of culprit flows from sketches without storing any explicit key information. No extra memory or memory accesses are needed for recording the streaming data. Meanwhile, the heavy change detection daemon runs in the background with space complexity and computational time sublinear to the key space size. This short paper describes the conceptual framework of the reversible sketches, as well as some initial approaches for implementation. See [23] for the optimized algorithms in details. comment We further apply various emph IP-mangling algorithms and emph bucket classification methods to reduce the false positives and false negatives. Evaluated with netflow traffic traces of a large edge router, we demonstrate that the reverse hashing can quickly infer the keys of culprit flows even for many changes with high accuracy.
61999607
Towards unbiased end-to-end network diagnosis
Internet fault diagnosis is extremely important for end users, overlay network service providers (like Akamai [1]) and even Internet service providers (ISPs). However, because link-level properties cannot be uniquely determined from end-to-end measurements, the accuracy of existing statistical diagnosis approaches is subject to uncertainty from statistical assumptions about the network. In this paper, we propose a novel Least-biased End-to-end Network Diagnosis (in short, LEND) system for inferring link-level properties like loss rate. We define a minimal identifiable link sequence (MILS) as a link sequence of minimal length whose properties can be uniquely identified from end-to-end measurements. We also design efficient algorithms to find all the MILSes and infer their loss rates for diagnosis. Our LEND system works for any network topology and for both directed and undirected properties, and incrementally adapts to network topology and property changes. It gives highly accurate estimates of the loss rates of MILSes, as indicated by both extensive simulations and Internet experiments. Furthermore, we demonstrate that such diagnosis can be achieved with fine granularity and in near real-time even for reasonably large overlay networks. Finally, LEND can supplement existing statistical inference approaches and provide smooth tradeoff between diagnosis accuracy and granularity.
887532
A Suite of Schemes for User-Level Network Diagnosis without Infrastructure
It is highly desirable and important for end users, with no special privileges, identify and pinpoint faults inside the network that degrade the performance of their applications. However, existing tools are inaccurate to infer the link-level loss rates and have large diagnosis granularity (in terms of the number of hops). To address these problems, we propose a suite of user-level diagnosis approaches in two categories: (1) only need to be deployed at the source and (2) deployed at both source and destination. For the former, we propose two fragmentation aided diagnosis approaches (FAD), Algebraic FAD and Opportunistic FAD, which uses IP fragmentation to enable accurate link-level loss rate inference. For the latter category, we propose Striped Probe Analysis (SPA) which significantly improves the diagnosis granularity over those of the source-only approaches. Internet experiments are applied to evaluate each individual schemes (including an improved version of the state-of-the-art tool, Tulip [1]) and various hybrid approaches. The results indicate that our approaches dramatically outperform existing work (especially for diagnosis granularity) and provide not only the best performance but also smooth tradeoff among deployment requirement, diagnosis accuracy and granularity.
166109
End-to-End Inference of Router Packet Forwarding Priority
Packet forwarding prioritization (PFP) in routers is one of the mechanisms commonly available to network administrators. PFP can have a significant impact on the performance of applications, the accuracy of measurement tools' results and the effectiveness of network troubleshooting procedures. Despite their potential impact, no information on PFP settings is readily available to end users. In this paper, we present an end-to-end approach for packet forwarding priority inference and its associated tool, POPI. This is the first attempt to infer router packet-forwarding priority through end-to-end measurement. Our POPI tool enables users to discover such network policies through the monitoring and rank classification of loss rates for different packet types. We validated our approach via statistical analysis, simulation, and wide-area experimentation in PlanetLab. As part of our wide-area experiments, we employed POPI to analyze 156 random paths across 162 PlanetLab nodes. We discovered 15 paths flagged with multiple priorities, 13 of which were further validated through hop-by-hop loss rates measurements. In addition, we surveyed all related network operators and received responses for about half of them confirming our inferences.
16325811
Collaborative detection and filtering of shrew DDoS attacks using spectral analysis
This paper presents a new spectral template-matching approach to countering shrew distributed denial-of-service (DDoS) attacks. These attacks are stealthy, periodic, pulsing, and low-rate in attack volume, very different from the flooding type of attacks. They are launched with high narrow spikes in very low frequency, periodically. Thus, shrew attacks may endanger the victim systems for a long time without being detected. In other words, such attacks may reduce the quality of services unnoticeably. Our defense method calls for collaborative detection and filtering (CDF) of shrew DDoS attacks. We detect shrew attack flows hidden in legitimate TCP/UDP streams by spectral analysis against pre-stored template of average attack spectral characteristics. This novel scheme is suitable for either software or hardware implementation. 
 
The CDF scheme is implemented with the NS-2 network simulator using real-life Internet background traffic mixed with attack datasets used by established research groups. Our simulated results show high detection accuracy by merging alerts from cooperative routers. Both theoretical modeling and simulation experimental results are reported here. The experiments achieved up to 95% successful detection of network anomalies along with a low 10% false positive alarms. The scheme cuts off malicious flows containing shrew attacks using a newly developed packet-filtering scheme. Our filtering scheme retained 99% of legitimate TCP flows, compared with only 20% TCP flows retained by using the Drop Tail algorithm. The paper also considers DSE FPGA, and network processor implementation issues and discusses limitations and further research challenges.
14235163
Robust controllability of interval fractional order linear time invariant systems
We consider uncertain fractional-order linear time invariant (FO-LTI) systems with interval coefficients. Our focus is on the robust controllability issue for interval FO-LTI systems in state-space form. We revisit the controllability problem for the case when there is no interval uncertainty. It turns out that the controllability check for FO-LTI systems amounts to checking the controllability of conventional integer order state space. Based on this fact, we further show that, for interval FO-LTI systems, the key is to check the linear dependency of a set of interval vectors. Illustrative examples are presented.
37291538
The Ensembl genome database project
The Ensembl (http://www.ensembl.org/) database project provides a bioinformatics framework to organise biology around the sequences of large genomes. It is a comprehensive source of stable automatic annotation of the human genome sequence, with confirmed gene predictions that have been integrated with external data sources, and is available as either an interactive web site or as flat files. It is also an open source software engineering project to develop a portable system able to handle very large genomes and associated requirements from sequence analysis to data storage and visualisation. The Ensembl site is one of the leading sources of human genome sequence annotation and provided much of the analysis for publication by the international human genome project of the draft genome. The Ensembl system is being installed around the world in both companies and academic sites on machines ranging from supercomputers to laptops.
894881
Performance improvement of TCP Vegas over heterogeneous networks
Current IP network has become the dominant paradigm for all networking environments. The significant cause of packet losses in such heterogenous networks is no longer limited to network congestion. Traditional TCP interprets every packet loss as caused by congestion which may be not the case in the current Internet. Misinterpretation of wireless random loss as an indication of network congestion results in TCP slowing down its sending rate unnecessarily. We propose a new variant of TCP Vegas named RedVegas. By using the innate nature of Vegas and congestion indications marked by routers, RedVegas may detect random packet losses precisely. Through the packet loss differentiation, RedVegas reacts appropriately to the losses, and therefore the throughput of connection over heterogeneous networks can be significantly improved.
17553402
Efficient Parallel Algorithms for Euclidean Distance Transform
The Euclidean Distance Transform (EDT) converts a binary image into one where each pixel has a value equal to its distance to the nearest foreground pixel. Two parallel algorithms for EDT transform on linear array with reconfigurable pipeline bus system (LARPBS) are presented. For an image with n×n pixels, the first algorithm can complete EDT transform in ) log log log log log log ( n n n O time using n processors. The second algorithm can compute the EDT in O( log n loglog n) time using n n log log 2 processors.