In a variety of disciplines such as social sciences, psychology, medicine and economics, the recorded data are considered to be noisy measurements of latent variables connected by some causal structure. This corresponds to a family of graphical models known as the structural equation model with latent variables. While linear non-Gaussian variants have been well-studied, inference in nonparametric structural equation models is still underdeveloped. We introduce a sparse Gaussian process parameterization that defines a non-linear structure connecting latent variables, unlike common formulations of Gaussian process latent variable models. The sparse parameterization is given a full Bayesian treatment without compromising Markov chain Monte Carlo efficiency. We compare the stability of the sampling procedure and the predictive ability of the model against the current practice.
Non-linear factor analysis has been studied for decades in the psychometrics literature Another instance of the whatever you do, somebody in psychometrics already did it long before'' law: http: www.stat.columbia.edu @math cook movabletype archives 2009 01 a .html . A review is provided by @cite_8 . However, most of the classic work is based on simple parametric models. A modern approach based on Gaussian processes is the Gaussian process latent variable model of @cite_17 . By construction, factor analysis cannot be used in applications where one is interested in learning functions relating latent variables, such as in causal inference. For embedding, factor analysis is easier to use and more robust to model misspecification than SEM analysis. Conversely, it does not benefit from well-specified structures and might be harder to interpret. @cite_15 discusses the interplay between factor analysis and SEM. Practical non-linear structural equation models are discussed by @cite_12 , but none of such approaches rely on nonparametric methods. Gaussian processes latent structures appear mostly in the context of dynamical systems (e.g., @cite_1 ). However, the connection is typically among data points only, not among variables within a data point, where on-line filtering is the target application.
@cite_8 @cite_1 @cite_15 @cite_12 @cite_17
2121739212 2016229608 2001667451 1716904016
Simple structure and other common principles of factor rotation do not in general provide strong grounds for attributing explanatory significance to the factors which they select. In contrast, it is shown that an extension of Cattell's principle of rotation to Proportional Profiles (PP) offers a basis for determining explanatory factors for three-way or higher order multi-mode data. Conceptual models are developed for two basic patterns of multi-mode data variation, systemand object-variation, and PP analysis is found to apply in the system-variation case. Although PP was originally formulated as a principle of rotation to be used with classic two-way factor analysis, it is shown to embody a latent three-mode factor model, which is here made explicit and generalized frown two to N "parallel occasions". As originally formulated, PP rotation was restricted to orthogonal factors. The generalized PP model is demonstrated to give unique "correct" solutions with oblique, non-simple structure, and even non-linear factor structures. A series of tests, conducted with synthetic data of known factor composition, demonstrate the capabilities of linear and non-linear versions of the model, provide data on the minimal necessary conditions of uniqueness, and reveal the properties of the analysis procedures when these minimal conditions are not fulfilled. In addition, a mathematical proof is presented for the uniqueness of the solution given certain conditions on the data. Three-mode PP factor analysis is applied to a three-way set of real data consisting of the fundamental and first three formant frequencies of 11 persons saying 8 vowels. A unique solution is extracted, consisting of three factors which are highly meaningful and consistent with prior knowledge and theory concerning vowel quality. The relationships between the three-mode PP model and Tucker's multi-modal model, McDonald's non-linear model and Carroll and Chang's multi-dimensional scaling model are explored. Factor analysis and its extensions are widely used in the social and behavioral sciences, and can be considered useful tools for exploration and model fitting in multivariate analysis. Despite its popu- larity in applications, factor analysis has attracted rather limited atten- tion from statisticians. Three issues, identification ambiguity, heavy reliance on normality, and limitation to linearity, may have contributed to statisticians' lack of interest in factor analysis. In this paper, the sta- tistical contributions to the first two issues are reviewed, and the third issue is addressed in detail. Linear models can be unrealistic even as an approximation in many applications, and often do not fit the data well without increasing the number of factors beyond the level explainable by the subject-matter theory. As an exploratory model, the conventional factor analysis model fails to address nonlinear structure underlying multivariate data. It is argued here that factor analysis does not need to be restricted to linearity and that nonlinear factor analysis can be formulated and carried out as a useful statistical method. In particular, for a general parametric nonlinear factor analysis model, the errors- in-variables parameterization is suggested as a sensible way to formu- late the model, and two procedures for model fitting are introduced and described. Tests for the goodness-of-fit of the model are also proposed. The procedures are studied through a simulation study. An example from personality testing is used to illustrate the issues and the methods. Determining interrelatedness structure of various entities from multiple time series data is of significant interest to many areas. Knowledge of such a structure can aid in identifying cause and effect relationships, clustering of similar entities, identification of representative elements and model reduction. The majority of existing results are based on correlation based indices which effectively assume a static relationship between the time series data and are not suitable for detecting interrelatedness when the time series are dynamically related or when the time series involve loops. In this paper, a methodology for identifying the interrelatedness structure of dynamically related time series data is presented that also allows for the presence of loops in the connectivity structure. A linear dynamic graph model is presented where it is assumed that each time series data is the sum of an independent stochastic noise source and a dynamically weighted sum of other time series data. A link is assumed to be present between two time series if the weight of a time series, which is a linear time-invariant filter, is nonzero in the formation of the other. Reconstruction of the link connectivity structure under various scenarios is considered. It is shown that when the linear dynamic graph is allowed to admit non-causal weights, then the links structure can be recovered with the possibility of identifying spurious connections. However, it is shown that the spurious links remain local, where, a spurious link is restricted to be within one hop of a true link. Furthermore, strategies for exact reconstruction of the link structure when the weights are restricted to be causal are developed. The main tools for determining the network topology are based on variations of Wiener filtering. A significant insight provided by the article is that, in the class of network models considered in the paper, the Wiener filter estimating a stochastic process based on other processes remains local in the sense that the Wiener filter utilizes only measurements local to the node being estimated. The Gaussian graphical model, a popular paradigm for studying relationship among variables in a wide range of applications, has attracted great attention in recent years. This paper considers a fundamental question: When is it possible to estimate low-dimensional parameters at parametric square-root rate in a large Gaussian graphical model? A novel regression approach is proposed to obtain asymptotically efficient estimation of each entry of a precision matrix under a sparseness condition relative to the sample size. When the precision matrix is not sufficiently sparse, or equivalently the sample size is not sufficiently large, a lower bound is established to show that it is no longer possible to achieve the parametric rate in the estimation of each entry. This lower bound result, which provides an answer to the delicate sample size question, is established with a novel construction of a subset of sparse precision matrices in an application of Le Cam's lemma. Moreover, the proposed estimator is proven to have optimal convergence rate when the parametric rate cannot be achieved, under a minimal sample requirement. The proposed estimator is applied to test the presence of an edge in the Gaussian graphical model or to recover the support of the entire model, to obtain adaptive rate-optimal estimation of the entire precision matrix as measured by the matrix @math operator norm and to make inference in latent variables in the graphical model. All of this is achieved under a sparsity condition on the precision matrix and a side condition on the range of its spectrum. This significantly relaxes the commonly imposed uniform signal strength condition on the precision matrix, irrepresentability condition on the Hessian tensor operator of the covariance matrix or the @math constraint on the precision matrix. Numerical results confirm our theoretical findings. The ROC curve of the proposed algorithm, Asymptotic Normal Thresholding (ANT), for support recovery significantly outperforms that of the popular GLasso algorithm.