Temporal Domain Generalization with Drift-Aware Dynamic Neural Network
Temporal domain generalization is a promising yet extremely challenging area
where the goal is to learn models under temporally changing data distributions
and generalize to unseen data distributions following the trends of the change.
The advancement of this area is challenged by: 1) characterizing data
distribution drift and its impacts on models, 2) expressiveness in tracking the
model dynamics, and 3) theoretical guarantee on the performance. To address
them, we propose a Temporal Domain Generalization with Drift-Aware Dynamic
Neural Network (DRAIN) framework. Specifically, we formulate the problem into a
Bayesian framework that jointly models the relation between data and model
dynamics. We then build a recurrent graph generation scenario to characterize
the dynamic graph-structured neural networks learned across different time
points. It captures the temporal drift of model parameters and data
distributions and can predict models in the future without the presence of
future data. In addition, we explore theoretical guarantees of the model
performance under the challenging temporal DG setting and provide theoretical
analysis, including uncertainty and generalization error. Finally, extensive
experiments on several real-world benchmarks with temporal drift demonstrate
the effectiveness and efficiency of the proposed method.