In the context of a motivating study of dynamic network flow data on a large-scale e-commerce web site, we develop Bayesian models for on-line sequential analysis for monitoring and adapting to changes reflected in node-node traffic. For large-scale networks, we customize core Bayesian time series analysis methods using dynamic generalized linear models (DGLMs) integrated into the multivariate network context using the concept of decouple recouple recently introduced in multivariate time series. This enables flexible dynamic modeling of flows on large-scale networks and exploitation of partial parallelization of analysis while critically maintaining coherence with an over-arching multivariate dynamic flow model. Development is anchored in a case-study on internet data, with flows of visitors to a commercial news web site defining a long time series of node-node counts on over 56,000 node pairs. Characterizing inherent stochasticity in traffic patterns, understanding node-node interactions, adapting to dynamic changes in flows and allowing for sensitive monitoring to flag anomalies are central questions. The methodology of dynamic network DGLMs will be of interest and utility in broad ranges of dynamic network flow studies.
This paper extends the recent work of @cite_6 in terms of network context and goals. Using an example of a small network (20 nodes), that work utilized a relatively simple smoothing model for latent time-varying Poisson rates on node-node pairs, and introduced the key idea of decouple recouple to enable scaling. While based on a state-space approach, the simple smoothing model is simply not able to adequately capture trends or abrupt changes in network flows, and the model structure is not flexible enough to incorporate other node flow features. Those challenges are addressed by DGLMs whose utility has been proven in other fields for many years [e.g.][Chapter 14] migon1985application,west1985dynamic,WestHarrison1997 .
@cite_6
2518177363
ABSTRACTTraffic flow count data in networks arise in many applications, such as automobile or aviation transportation, certain directed social network contexts, and Internet studies. Using an example of Internet browser traffic flow through site-segments of an international news website, we present Bayesian analyses of two linked classes of models which, in tandem, allow fast, scalable, and interpretable Bayesian inference. We first develop flexible state-space models for streaming count data, able to adaptively characterize and quantify network dynamics efficiently in real-time. We then use these models as emulators of more structured, time-varying gravity models that allow formal dissection of network dynamics. This yields interpretable inferences on traffic flow characteristics, and on dynamics in interactions among network nodes. Bayesian monitoring theory defines a strategy for sequential model assessment and adaptation in cases when network flow data deviate from model-based predictions. Exploratory...