Temporal point process is an expressive tool for modeling event sequences over time. In this paper, we take a reinforcement learning view whereby the observed sequences are assumed to be generated from a mixture of latent policies. The purpose is to cluster the sequences with different temporal patterns into the underlying policies while learning each of the policy model. The flexibility of our model lies in: i) all the components are networks including the policy network for modeling the intensity function of temporal point process; ii) to handle varying-length event sequences, we resort to inverse reinforcement learning by decomposing the observed sequence into states (RNN hidden embedding of history) and actions (time interval to next event) in order to learn the reward function, thus achieving better performance or increasing efficiency compared to existing methods using rewards over the entire sequence such as log-likelihood or Wasserstein distance. We adopt an expectation-maximization framework with the E-step estimating the cluster labels for each sequence, and the M-step aiming to learn the respective policy. Extensive experiments show the efficacy of our method against state-of-the-arts.
Traditional TPP models are mostly developed around the design of the intensity function @math which measures the instantaneous event occurrence rate at time @math , like Reinforced Poisson processes @cite_13 , Self-exciting process (Hawkes process) @cite_8 , Reactive point process @cite_16 , etc. An obvious limitation of these traditional models is that they all assume all the samples obey a single parametric form which is too idealistic for real-world data. This also suggests the need for learning clustered behaviors beyond single model based methods.
@cite_16 @cite_13 @cite_8
2090320383 2145037371 138372711
Massachusetts Institute of Technology and the University of Washington Reactive point processes (RPPs) are a new statistical model designed for predicting discrete events in time, based on past history. RPPs were developed to handle an important problem within the domain of electrical grid reliability: short term prediction of electrical grid failures (“manhole events”), including outages, fires, explosions, and smoking manholes, which can cause threats to public safety and reliability of electrical service in cities. RPPs incorporate self-exciting, self-regulating, and saturating components. The self-excitement occurs as a result of a past event, which causes a temporary rise in vulnerability to future events. The self-regulation occurs as a result of an external inspection which temporarily lowers vulnerability to future events. RPPs can saturate when too many events or inspections occur close together, which ensures that the probability of an event stays within a realistic range. Two of the operational challenges for power companies are i) making continuous-time failure predictions, and ii) cost benefit analysis for decision making and proactive maintenance. RPPs are naturally suited for handling both of these challenges. We use the model to predict power-grid failures in Manhattan over a short term horizon, and use to provide a cost benefit analysis of different proactive maintenance programs. The models surveyed include generalized Polya urns, reinforced random walks, interacting urn models, and continuous reinforced processes. Emphasis is on methods and results, with sketches provided of some proofs. Applications are discussed in statistics, biology, economics and a number of other areas. IN contagious processes (e.g. measles, hijacking, etc.) the occurrence of events increases the probability of further events occurring in the near future. Also several series of events may interact with each other, for example one might consider notifications of some disease in a number of adjacent regions which would interact through infectives or carriers moving between the regions. In this paper we postulate a model for such processes and derive a general expression for the point spectral matrices. These theoretical spectra are useful for comparison with spectra estimated from data and thus provide a means of evaluating the fit of such a model in the manner of Bartlett (1963). The model studied was put forward in an earlier paper (Hawkes, 1971) but the solution was obtained only in special cases. In this paper an elegant solution is obtained for the. general case. Consider a stationary k-variate point process N(t), where Ni(t) represents the cumulative number of events in the ith process up to time t, with intensity vector X = of dN(t) dt and covariance density matrix