An extensive body of research deals with estimating the correlation and the Hurst parameter of Internet traffic traces. The significance of these statistics is due to their fundamental impact on network performance. The coverage of Internet traffic traces is, however, limited since acquiring such traces is challenging with respect to, e.g., confidentiality, logging speed, and storage capacity. In this work, we investigate how the correlation of Internet traffic can be reliably estimated from random traffic samples. These samples are observed either by passive monitoring within the network, or otherwise by active packet probes at end systems. We analyze random sampling processes with different inter-sample distributions and show how to obtain asymptotically unbiased estimates from these samples. We quantify the inherent limitations that are due to limited observations and explore the influence of various parameters, such as sampling intensity, network utilization, or Hurst parameter on the estimation accuracy. We design an active probing method which enables simple and lightweight traffic sampling without support from the network. We verify our approach in a controlled network environment and present comprehensive Internet measurements. We find that the correlation exhibits properties such as long range dependence as well as periodicities and that it differs significantly across Internet paths and observation times.
Two important aspects concerning network probing are the measurement intrusiveness and the interaction of probes with the measured system. The first aspect is usually addressed by minimizing the probing rate while controlling the quality of the results. The second aspect is more involved, since the probes perturb the system leading to distorted observations. For example, measuring queueing delays of probes to determine the true queue length distribution is governed by a type of Heisenberg uncertainty @cite_24 , since the probes alter the queue length. The authors describe the impact of the probing intensity on the accuracy of the result using the notion of asymptotic variance. The effect is increased in case of LRD traffic, although not given in closed form, leading to higher uncertainty in the estimated waiting time @cite_24 .
@cite_24
2158778126 2098214968 1487153207 2159326294
Packet delay and loss are two fundamental measures of performance. Using active probing to measure delay and loss typically involves sending Poisson probes, on the basis of the PASTA property (Poisson Arrivals See Time Averages), which ensures that Poisson probing yields unbiased estimates. Recent work, however, has questioned the utility of PASTA for probing and shown that, for delay measurements, i) a wide variety of processes other than Poisson can be used to probe with zero bias and ii) Poisson probing does not necessarily minimize the variance of delay estimates. In this paper, we determine optimal probing processes that minimize the mean-square error of measurement estimates for both delay and loss. Our contributions are twofold. First, we show that a family of probing processes, specifically Gamma renewal probing processes, has optimal properties in terms of bias and variance. The optimality result is general, and only assumes that the target process we seek to optimally measure via probing, such as a loss or delay process, has a convex auto-covariance function. Second, we use empirical datasets to demonstrate the applicability of our results in practice, specifically to show that the convexity condition holds true and that Gamma probing is indeed superior to Poisson probing. Together, these results lead to explicit guidelines on designing the best probe streams for both delay and loss estimation. Active probes of network performance represent samples of the underlying performance of a system. Some effort has gone into considering appropriate sampling patterns for such probes, i.e., there has been significant discussion of the importance of sampling using a Poisson process to avoid biases introduced by synchronization of system and measurements. However, there are unanswered questions about whether Poisson probing has costs in terms of sampling efficiency, and there is some misinformation about what types of inferences are possible with different probe patterns. This paper provides a quantitative comparison of two different sampling methods. This paper also shows that the irregularity in probing patterns is useful not just in avoiding synchronization, but also in determining frequency-domain properties of a system. This paper provides a firm basis for practitioners or researchers for making decisions about the type of sampling they should use in a particular applications, along with methods for the analysis of their outputs In this paper we address the problem of online bandwidth estimation in wired and wireless LANs. To this end, we employ active probing, i.e. we continuously inject packet probes into the network. We present the key challenges and analyze the trade-offs between fast change detection and estimate smoothness. We show the benefit of using Kalman filtering to obtain optimal estimates under certain conditions and provide a procedure for parameterizing the filter with respect to specific use cases. Furthermore, we evaluate the influence of probing train length on the results. Based on our findings we developed a tool implementing the presented methodology. We support our theoretical results by a number of real-world measurements as well as simulations. Despite many efforts over the past decade, the ability to generate topological maps of the Internet at the router-level accurately and in a timely fashion remains elusive. Mapping campaigns commonly involve traceroute-like probing that are usually non-adaptive and incomplete, thus revealing only a portion of the underlying topology. In this paper we demonstrate that standard probing methods yield datasets that implicitly contain information about much more than just the directly observed links and routers. Each probe, in addition to the underlying domain knowledge, returns information that places constraints on the underlying topology, and by integrating a large number of such constraints it is possible to accurately infer the existence of unseen components of the Internet. We describe DomainImpute, a novel data analysis methodology designed to accurately infer the unseen hop-count distances between observed routers. We use both synthetic and a large empirical dataset to validate the proposed methods. On our empirical real world dataset, we show that our methods can estimate over 55 of the unseen distances between observed routers to within a one-hop error.