Probabilistic graphical models compactly represent joint distributions by decomposing them into factors over subsets of random variables. In Bayesian networks, the factors are conditional probability distributions. For many problems, common information exists among those factors. Adding similarity restrictions can be viewed as imposing prior knowledge for model regularization. With proper restrictions, learned models usually generalize better. In this work, we study methods that exploit such high-level similarities to regularize the learning process and apply them to the task of modeling the wave propagation in inhomogeneous media. We propose a novel distribution-based penalization approach that encourages similar conditional probability distribution rather than force the parameters to be similar explicitly. We show in experiment that our proposed algorithm solves the modeling wave propagation problem, which other baseline methods are not able to solve.
Multi-Task Learning (MTL) is a learning paradigm in machine learning and its target is to leverage useful information contained in multiple related tasks to help improve the generalization performance of all the tasks. Consider the wave propagation problem, we can formulize it as finding the distribution @math , where @math denotes the state of all nodes at time @math . The assumption is that the state of each node at time @math is independent of other nodes at time @math given the states of its neighbors at time @math . So the distribution can be decomposed into @math (assume we have @math nodes in total) conditional distributions. Rather than using @math neural networks to approximate those distributions, MTL uses one neural network with @math outputs. The first few hidden layers are shared among nodes, while the following layers are node-specific. @cite_9 shows that such setting greatly reduces the risk of overfitting. This makes sense intuitively: The more factors we are learning simultaneously, the more our model has to find a representation that captures all of the factors and the less is the chance of overfitting. We implement a MTL model as the opponent of our distribution-based penalization model.
@cite_9
2143419558
We study network-design problems with multiple design objectives. In particular, we look at two cost measures to be minimized simultaneously: the total cost of the network and the maximum degree of any node in the network. Our main result can be roughly stated as follows: given an integer @math , we present approximation algorithms for a variety of network-design problems on an @math -node graph in which the degree of the output network is @math and the cost of this network is @math times that of the minimum-cost degree- @math -bounded network. These algorithms can handle costs on nodes as well as edges. Moreover, we can construct such networks so as to satisfy a variety of connectivity specifications including spanning trees, Steiner trees and generalized Steiner forests. The performance guarantee on the cost of the output network is nearly best-possible unless @math . We also address the special case in which the costs obey the triangle inequality. In this case, we obtain a polynomial-time approximation algorithm with a stronger performance guarantee. Given a bound @math on the degree, the algorithm finds a degree- @math -bounded network of cost at most a constant time optimum. There is no algorithm that does as well in the absence of triangle inequality unless @math . We also show that in the case of spanning networks, we can simultaneously approximate within a constant factor yet another objective: the maximum cost of any edge in the network, also called the bottleneck cost of the network. We extend our algorithms to find TSP tours and @math -connected spanning networks for any fixed @math that simultaneously approximate all these three cost measures.