This article is an extended version of previous work of the authors [40, 41] on low-rank matrix estimation in the presence of constraints on the factors into which the matrix is factorized. Low-rank matrix factorization is one of the basic methods used in data analysis for unsupervised learning of relevant features and other types of dimensionality reduction. We present a framework to study the constrained low-rank matrix estimation for a general prior on the factors, and a general output channel through which the matrix is observed. We draw a paralel with the study of vector-spin glass models - presenting a unifying way to study a number of problems considered previously in separate statistical physics works. We present a number of applications for the problem in data analysis. We derive in detail a general form of the low-rank approximate message passing (Low- RAMP) algorithm, that is known in statistical physics as the TAP equations. We thus unify the derivation of the TAP equations for models as different as the Sherrington-Kirkpatrick model, the restricted Boltzmann machine, the Hopfield model or vector (xy, Heisenberg and other) spin glasses. The state evolution of the Low-RAMP algorithm is also derived, and is equivalent to the replica symmetric solution for the large class of vector-spin glass models. In the section devoted to result we study in detail phase diagrams and phase transitions for the Bayes-optimal inference in low-rank matrix estimation. We present a typology of phase transitions and their relation to performance of algorithms such as the Low-RAMP or commonly used spectral methods.
It has become a tradition in related literature @cite_53 to conjecture that the performance of the Low-RAMP algorithm cannot be improved by other polynomial algorithms. We do analyze here in detail the cases where Low-RAMP does not achieve the MMSE, and we remark that since effects of replica symmetry breaking need to be taken into account when evaluating the performance of the best polynomial algorithms, the conjecture of the Low-RAMP optimality among the polynomial algorithms deserves further detailed investigation.
@cite_53
2116394223 2172185514 2536849891 1825819013
We present a theory, based on statistical mechanics, to evaluate analytically the performance of uncoded, fully synchronous, randomly spread code-division multiple-access (CDMA) multiuser detectors with additive white Gaussian noise (AWGN) channel, under perfect power control, and in the large-system limit. Application of the replica method, a tool developed in the literature of statistical mechanics, allows us to derive analytical expressions for the bit-error rate, as well as the multiuser efficiency, of the individually optimum (IO) and jointly optimum (JO) multiuser detectors over the whole range of noise levels. The information-theoretic capacity of the randomly spread CDMA channel and the performance of decorrelating and linear minimum mean-square error (MMSE) detectors are also derived in the same replica formulation, thereby demonstrating validity of the statistical-mechanical approach. In this paper, we propose an optimum algorithm, in the minimum mean-square-error (mmse) sense, for panchromatic (Pan) sharpening of very high resolution multispectral (MS) images. The solution minimizes the squared error between the original MS image and the fusion result obtained by spatially enhancing a degraded version of the MS image through a degraded version, by the same scale factor, of the Pan image. The fusion result is also optimal at full scale under the assumption of invariance of the fusion parameters across spatial scales. The following two versions of the algorithm are presented: a local mmse (lmmse) solution and a fast implementation which globally optimizes the fusion parameters with a moderate performance loss with respect to the lmmse version. We show that the proposed method is computationally practical, even in the case of local optimization, and it outperforms the best state-of-the-art Pan-sharpening algorithms, as resulted from the IEEE Data Fusion Contest 2006, on true Ikonos and QuickBird data and on simulated Pleiades data. We study the recovery of sparse vectors from subsampled random convolutions via @math -minimization. We consider the setup in which both the subsampling locations as well as the generating vector are chosen at random. For a subgaussian generator with independent entries, we improve previously known estimates: if the sparsity @math is small enough, i.e., @math , we show that @math measurements are sufficient to recover @math -sparse vectors in dimension @math with high probability, matching the well-known condition for recovery from standard Gaussian measurements. If @math is larger, then essentially @math measurements are sufficient, again improving over previous estimates. Our results are shown via the so-called robust null space property which is weaker than the standard restricted isometry property. Our method of proof involves a novel combination of small ball estimates with chaining techniques which should be of independent interest. The following question is due to Chatterjee and Varadhan 2011. Fix 0<p<r<1 and take G Gn,p, the Erdi¾?s-Renyi random graph with edge density p, conditioned to have at least as many triangles as the typical Gn,r. Is G close in cut-distance to a typical Gn,r? Via a beautiful new framework for large deviation principles in Gn,p, Chatterjee and Varadhan gave bounds on the replica symmetric phase, the region of p,r where the answer is positive. They further showed that for any small enough p there are at least two phase transitions as r varies. We settle this question by identifying the replica symmetric phase for triangles and more generally for any fixed d-regular graph. By analyzing the variational problem arising from the framework of Chatterjee and Varadhan we show that the replica symmetry phase consists of all p,r such that rd,hpr lies on the convex minorant of xi¾?hpx1 d where hp is the rate function of a binomial with parameter p. In particular, the answer for triangles involves hpx rather than the natural guess of hpx1 3 where symmetry was previously known. Analogous results are obtained for linear hypergraphs as well as the setting where the largest eigenvalue of G Gn,p is conditioned to exceed the typical value of the largest eigenvalue of Gn,r. Building on the work of Chatterjee and Diaconis 2012 we obtain additional results on a class of exponential random graphs including a new range of parameters where symmetry breaking occurs. En route we give a short alternative proof of a graph homomorphism inequality due to Kahn 2001 and Galvin and Tet ali 2004. © 2014 Wiley Periodicals, Inc. Random Struct. Alg., 47, 109-146, 2015