This paper studies the structure of several real-world traces (including Facebook, High-Performance Computing, Machine Learning, and simulation generated traces) and presents a systematic approach to quantify and compare the structure of packet traces based on the entropy contained in the trace file. Insights into the structure of packet traces can lead to improved network algorithms that are optimized toward specific traffic patterns. We then present a methodology to quantify the temporal and non-temporal components of entropy contained in a packet trace, called the trace complexity, using randomization and compression. We show that trace complexity provides unique insights into the characteristics of various applications and argue that there is a need for traffic generation models that preserve the intrinsic structure of empirically measured application traces. We then propose a traffic generator model that is able to produce a synthetic trace that matches the complexity level of its corresponding real-world trace.
Information-theoretic approaches and compression methodologies have already been proven successful in capturing entropy in other domains such as email @cite_10 , or comment @cite_28 spam filtering, or estimating neural discharges @cite_11 . The study of traffic patterns and the design of models is an evergreen topic of high relevance in the networking literature, and examples where measurement studies spurred much research into traffic modeling dates back to the 1990s @cite_38 @cite_42 @cite_33 . Since then, a large number of methodologies have been developed @cite_20 @cite_12 @cite_32 , based on temporal statistics @cite_17 @cite_3 @cite_31 , spatial statistics @cite_43 @cite_36 , and physical @cite_13 and information-theoretic @cite_7 @cite_39 models. In contrast to prior work, we are primarily interested in the communication pattern itself, rather than in the volume or headers of the exchanged data. While our work builds upon many significant results developed over the last decades @cite_34 , we are not aware of any work which allows to systematically differentiate between temporal and non-temporal components of traffic traces.
@cite_13 @cite_38 @cite_33 @cite_7 @cite_28 @cite_36 @cite_42 @cite_32 @cite_17 @cite_3 @cite_39 @cite_43 @cite_31 @cite_34 @cite_10 @cite_12 @cite_20 @cite_11
2094331586 40609341 69989483 2903103701
Understanding and solving performance-related issues of current and future (5G+) networks requires the availability of realistic, yet simple and manageable, traffic models which capture and regenerate various properties of real traffic with sufficient accuracy and minimum number of parameters. Traffic in wireless cellular networks must be modeled in the space domain as well as the time domain. Modeling traffic in the time domain has been investigated well. However, for modeling the User Equipment (UE) distribution in the space domain, either the unrealistic uniform Poisson model, or some non-adjustable model, or specifc data from operators, is commonly used. In this paper, stochastic geometry is used to explain the similarities of traffic modeling in the time domain and the space domain. It is shown that traffic modeling in the time domain is a special one-dimensional case of traffic modeling in the space domain. Unified and non-parameterized metrics for characterizing the heterogeneity of traffic in the time domain and the space domain are proposed and their equivalence to the inter-arrival time, a well accepted metric in the time domain, is demonstrated. Coefficient of Variation (CoV), the normalized second-order statistic, is suggested as an appropriate statistical property of traffic to be measured. Simulation results show that the proposed metrics capture the properties of traffic more accurately than the existing metrics. Finally, the performance of LTE networks under modeled traffic using the new metrics is illustrated. The traffic networks reflect the pulse and structure of a city and shows some dynamic characteristic. Previous research in mining structure from networks mostly focus on static networks and fail to exploit the temporal patterns. In this paper, we aim to solve the problem of discovering the urban spatio-temporal structure from time-evolving traffic networks. We model the time-evolving traffic networks into a 3-order tensor, each element of which indicates the volume of traffic from i-th origin area to j-th destination area in k-th time domain. Considering traffic data and urban contextual knowledge together, we propose a regularized Non-negative Tucker Decomposition (rNTD) method, which discovers the spatial clusters, temporal patterns and relations among them simultaneously. Abundant experiments are conducted in a large dataset collected from Beijing. Results show that our method outperforms the baseline method. Numerous massive data sets, ranging from flows of Internet traffic to logs of supermarket transactions, have emerged during the past few years. Their overwhelming size and the typically restricted access to them call for new computational models. This thesis studies three such models: sampling computations, data stream computations, and sketch computations. While most of the previous work focused on designing algorithms in the new models, this thesis revolves around the limitations of the models. We develop a suite of lower bound techniques that characterize the complexity of functions in these models, indicating which problems can be solved efficiently in them. We derive specific bounds for a multitude of practical problems, arising from applications in database, networking, and information retrieval, such as frequency statistics, selection functions, statistical moments, and distance estimation. We present general, powerful, and easy to use lower bound techniques for the sampling model. The techniques apply to all functions and address both oblivious and adaptive sampling. They frequently produce optimal bounds for a wide range of functions. They are stated in terms of new combinatorial and statistical properties of functions, which are easy to calculate. We obtain lower bounds for the data stream and sketch models through one-way and simultaneous communication complexity. We develop lower bounds for the latter via a new information-theoretic view of communication complexity. A highlight of this work is an optimal simultaneous communication complexity lower bound for the important multi-party set-disjointness problem. Finally, we present a powerful method for proving lower bounds for general communication complexity. The method is based on a direct sum property of a new measure of complexity for communication complexity protocols and on a novel statistical view of communication complexity. We use the technique to obtain improved communication complexity and data stream lower bounds for several problems, including multi-party set-disjointness, frequency moments, and Lp distance estimation. These results solve open problems of Alon, Matias, and Szegedy and of Saks and Sun. In this paper, we make the first attempt to build a framework to simultaneously estimate semantic parts, shape, translation, and orientation of cars from single street view. Our framework contains three major contributions. Firstly, a novel domain adaptation approach based on the class consistency loss is developed to transfer our part segmentation model from the synthesized images to the real images. Secondly, we propose a novel network structure that leverages part-level features from street views and 3D losses for pose and shape estimation. Thirdly, we construct a high quality dataset that contains more than 300 different car models with physical dimensions and part-level annotations based on global and local deformations. We have conducted experiments on both synthesized data and real images. Our results show that the domain adaptation approach can bring 35.5 percentage point performance improvement in terms of mean intersection-over-union score (mIoU) comparing with the baseline network using domain randomization only. Our network for translation and orientation estimation achieves competitive performance on highly complex street views (e.g., 11 cars per image on average). Moreover, our network is able to reconstruct a list of 3D car models with part-level details from street views, which could benefit various applications such as fine-grained car recognition, vehicle re-identification, and traffic simulation.