In this paper, we aim to provide a systematic study of the relationship between Chernoff information and topological, as well as algebraic properties of the corresponding Gaussian tree graphs for the underlying graphical testing problems. We first show the relationship between Chernoff information and generalized eigenvalues of the associated covariance matrices. It is then proved that Chernoff information between two Gaussian trees sharing certain local subtree structures can be transformed into that of two smaller trees. Under our proposed grafting operations, bottleneck Gaussian trees, namely, Gaussian trees connected by one such operation, can thus be simplified into two 3-node Gaussian trees, whose topologies and edge weights are subject to the specifics of the operation. Thereafter, we provide a thorough study about how Chernoff information changes when small differences are accumulated into bigger ones via concatenated grafting operations. It is shown that the two Gaussian trees connected by more than one grafting operation may not have bigger Chernoff information than that of one grafting operation unless these grafting operations are separate and independent. At the end, we propose an optimal linear dimensional reduction method related to generalized eigenvalues.
Traditional dimension reduction methods, such as Principal Component Analysis (PCA) and other Representation Learning @cite_14 , aim to find the optimal features with maximum information. They only consider a distribution at one time and don't care how we use the resulting low-dimensional data. But our method considers two hypothesis at the same time and want to keep the most information for classification. Some features may be important in PCA, but invalid in our method because these features in two hypothesis are similar. Our objective function here is the Chernoff information between the resulting low-dimensional distributions. We linearly transform an @math dimensional Gaussian vector @math to an @math dimensional vector @math , through an @math matrix @math . We provide an optimal linear transformation @math which can maximize Chernoff information in the classification of two hypothesis.
@cite_14
2163922914
The success of machine learning algorithms generally depends on data representation, and we hypothesize that this is because different representations can entangle and hide more or less the different explanatory factors of variation behind the data. Although specific domain knowledge can be used to help design representations, learning with generic priors can also be used, and the quest for AI is motivating the design of more powerful representation-learning algorithms implementing such priors. This paper reviews recent work in the area of unsupervised feature learning and deep learning, covering advances in probabilistic models, autoencoders, manifold learning, and deep networks. This motivates longer term unanswered questions about the appropriate objectives for learning good representations, for computing representations (i.e., inference), and the geometrical connections between representation learning, density estimation, and manifold learning.