Informed Learning by Wide Neural Networks: Convergence, Generalization and Sampling Complexity
By integrating domain knowledge with labeled samples, informed machine
learning has been emerging to improve the learning performance for a wide range
of applications. Nonetheless, rigorous understanding of the role of injected
domain knowledge has been under-explored. In this paper, we consider an
informed deep neural network (DNN) with over-parameterization and domain
knowledge integrated into its training objective function, and study how and
why domain knowledge benefits the performance. Concretely, we quantitatively
demonstrate the two benefits of domain knowledge in informed learning -
regularizing the label-based supervision and supplementing the labeled samples
- and reveal the trade-off between label and knowledge imperfectness in the
bound of the population risk. Based on the theoretical analysis, we propose a
generalized informed training objective to better exploit the benefits of
knowledge and balance the label and knowledge imperfectness, which is validated
by the population risk bound. Our analysis on sampling complexity sheds lights
on how to choose the hyper-parameters for informed learning, and further
justifies the advantages of knowledge informed learning.