An extensive body of research deals with estimating the correlation and the Hurst parameter of Internet traffic traces. The significance of these statistics is due to their fundamental impact on network performance. The coverage of Internet traffic traces is, however, limited since acquiring such traces is challenging with respect to, e.g., confidentiality, logging speed, and storage capacity. In this work, we investigate how the correlation of Internet traffic can be reliably estimated from random traffic samples. These samples are observed either by passive monitoring within the network, or otherwise by active packet probes at end systems. We analyze random sampling processes with different inter-sample distributions and show how to obtain asymptotically unbiased estimates from these samples. We quantify the inherent limitations that are due to limited observations and explore the influence of various parameters, such as sampling intensity, network utilization, or Hurst parameter on the estimation accuracy. We design an active probing method which enables simple and lightweight traffic sampling without support from the network. We verify our approach in a controlled network environment and present comprehensive Internet measurements. We find that the correlation exhibits properties such as long range dependence as well as periodicities and that it differs significantly across Internet paths and observation times.
Further, the authors of @cite_29 establish general conditions, such that Arrivals See Time Averages (ASTA) holds, i.e., bias free estimates are not limited to Poisson sampling. In a recent work the authors of @cite_36 coined the term NIMASTA, i.e. Non-intrusive Mixing Arrivals See Time Averages, in the context of network measurements. Using an argument on joint ergodicity, the authors prove an almost sure convergence of where @math is a sample of the process @math at time @math and @math is a general positive function of @math . The sampling times @math for @math are chosen according to a sampling process. The target metric is specified depending on the chosen function @math . Eq. is satisfied when the process @math is ergodic and the sampling process is mixing @cite_36 . The authors in @cite_0 show that Poisson sampling, though bias free, does not guarantee minimum variance estimates.
@cite_36 @cite_29 @cite_0
2032237051 1983670906 2582952306 2122735658
We investigate when Arrivals See Time Averages ASTA in a stochastic model; i.e., when the stationary distribution of an embedded sequence, obtained by observing a continuous-time stochastic process just prior to the points arrivals of an associated point process, coincides with the stationary distribution of the observed process. We also characterize the relation between the two distributions when ASTA does not hold. We introduce a Lack of Bias Assumption LBA which stipulates that, at any time, the conditional intensity of the point process, given the present state of the observed process, be independent of the state of the observed process. We show that LBA, without the Poisson assumption, is necessary and sufficient for ASTA in a stationary process framework. Consequently, LBA covers known examples of non-Poisson ASTA, such as certain flows in open Jackson queueing networks, as well as the familiar Poisson case PASTA. We also establish results to cover the case in which the process is observed just after the points, e.g., when departures see time averages. Finally, we obtain a new proof of the Arrival Theorem for product-form queueing networks. The well-known PASTA ("Poisson Arrivals See Time Averages") property states that, under very general conditions, the fraction of Poisson arrivals that observe an underlying process in a particular state is equal, asymptotically, to the fraction of time the process spends in that state. When applied to network inference, PASTA implies that a Poisson probing stream provides an unbiased estimate of the desired time average. Our objective is to examine the practical significance of the PASTA property in the context of realistic RTT, loss rate and packet pair dispersion measurements with a finite (but not small) number of samples. In particular, we first evaluate the differences between the point estimates (median RTT, loss rate, and median dispersion) that result from Poisson and Periodic probing. Our evaluation is based on a rich set of measurements between 23 PlanetLab hosts. The experimental results show that in almost all measurement sessions the differences between the Poisson and Periodic point estimates are insignificant. In the case of RTT and dispersion measurements, we also used a non-parametric goodness-of-fit test, based on the Kullback-Leibler distance, to evaluate the similarity of the distributions that result from Poisson and Periodic probing. The results show that in more than 90 of the measurements there is no statistically significant difference between the two distributions. In this paper, we consider a problem of sampling a Wiener process, with samples forwarded to a remote estimator via a channel with queueing and random delay. The estimator reconstructs a real-time estimate of the signal from causally received samples. Motivated by recent research on Age-of-Information, we study the optimal sampling strategy for minimizing the mean square estimation error subject to a sampling-rate constraint. We prove that the optimal sampling strategy is a threshold policy, and find the optimal threshold. This threshold is determined by the sampling-rate constraint and how much the Wiener process varies during the channel delay. An unexpected consequence is that even in the absence of the sampling-rate constraint, the optimal strategy is not zero-wait sampling in which a new sample is taken once the previous sample is delivered; rather, there is a non-zero waiting time before taking the next sample. Further, if the sampling times are independent of the observed Wiener process, the optimal sampling problem reduces to an age-of-information optimization problem that has been recently solved. Our comparisons show that the estimation error of the optimal sampling policy is much smaller than those of age-optimal sampling, zero-wait sampling, and classic uniform sampling. A constructive solution of the irregular sampling problem for band- limited functions is given. We show how a band-limited function can be com- pletely reconstructed from any random sampling set whose density is higher than the Nyquist rate, and give precise estimates for the speed of convergence of this iteration method. Variations of this algorithm allow for irregular sampling with derivatives, reconstruction of band-limited functions from local averages, and irregular sampling of multivariate band-limited functions. In the irregular sampling problem one is asked whether and how a band- limited function can be completely reconstructed from its irregularly sam- pled values f(xi). This has many applications in signal and image processing, seismology, meteorology, medical imaging, etc. Finding constructive solutions of this problem has received considerable attention among mathematicians and engineers. The mathematical literature provides several uniqueness results (1, 2, 17, 18, 19). It is now part of the folklore that for stable sampling the sampling rate must be at least the Nyquist rate (18). These results, as deep as they are, have had little impact for the applied sciences, because they were not constructive. If the sampling set is just a perturbation of the regular oversampling, then a reconstruction method has been obtained in a seminal paper by Duffin and Schaeffer (6) (see also (29)): if for some L > 0, a > 0, and o > 0 the sampling points xk , k e Z , satisfy (a) - ok a, k ^ I, then the norm equivalence A iR (x) ) with w < n o. This norm equivalence implies that it is possible to reconstruct through an iterative procedure, the so-called frame method. Most of the later work on constructive methods consists of variations of this method (3, 21, 22, 26). The above conditions on the sampling set exclude random irregular sampling sets, e.g., sets with regions of higher sampling density. A partial, but undesirable remedy, to handle highly irregular sampling sets, would be to force the above conditions by throwing away information on part of the points and accept a very slow convergence of the iteration.