Markov random fields are used to model high dimensional distributions in a number of applied areas. Much recent interest has been devoted to the reconstruction of the dependency structure from independent samples from the Markov random fields. We analyze a simple algorithm for reconstructing the underlying graph defining a Markov random field on @math nodes and maximum degree @math given observations. We show that under mild non-degeneracy conditions it reconstructs the generating graph with high probability using @math samples where @math depend on the local interactions. For most local interaction @math are of order @math . Our results are optimal as a function of @math up to a multiplicative constant depending on @math and the strength of the local interactions. Our results seem to be the first results for general models that guarantee that the generating model is reconstructed. Furthermore, we provide explicit @math running time bound. In cases where the measure on the graph has correlation decay, the running time is @math for all fixed @math . We also discuss the effect of observing noisy samples and show that as long as the noise level is low, our algorithm is effective. On the other hand, we construct an example where large noise implies non-identifiability even for generic noise and interactions. Finally, we briefly show that in some simple cases, models with hidden nodes can also be recovered.
The same problem as in the present work (but restricted to the Ising model) was studied by Wainwright, @cite_8 , where an algorithm based on @math -regularization was introduced. The algorithm presented is efficient also for dense graphs with running time @math but is applicable only in very restricted settings. The work only applies to the Ising model and more importantly only models with edge interactions (no larger cliques are allowed). The most important restrictions are the two conditions in the paper (A1 and A2). Condition A1 requires (among other things) that the covariates [spins] do not become overly dependent''. Verifying when the conditions holds seems hard. However, it is easy to see that this condition fails for standard models such as the Ising model on the lattice or on random @math -regular graphs when the model is at low temperatures, i.e. for @math in the case of the two dimensional Ising model and @math for random @math -regular graphs.
@cite_8
1983272724 2090216341 1977990557 2215999167
The individual spins of the Ising model are assumed to interact with an external agency (e.g., a heat reservoir) which causes them to change their states randomly with time. Coupling between the spins is introduced through the assumption that the transition probabilities for any one spin depend on the values of the neighboring spins. This dependence is determined, in part, by the detailed balancing condition obeyed by the equilibrium state of the model. The Markoff process which describes the spin functions is analyzed in detail for the case of a closed N‐member chain. The expectation values of the individual spins and of the products of pairs of spins, each of the pair evaluated at a different time, are found explicitly. The influence of a uniform, time‐varying magnetic field upon the model is discussed, and the frequency‐dependent magnetic susceptibility is found in the weak‐field limit. Some fluctuation‐dissipation theorems are derived which relate the susceptibility to the Fourier transform of the tim... We establish tight results for rapid mixing of Gibbs samplers for the Ferromagnetic Ising model on general graphs. We show that if (d−1) tanh β < 1, then there exists a constant C such that the discrete time mixing time of Gibbs samplers for the ferromagnetic Ising model on any graph of n vertices and maximal degree d, where all interactions are bounded by β, and arbitrary external fields are bounded by Cnlogn. Moreover, the spectral gap is uniformly bounded away from 0 for all such graphs, as well as for infinite graphs of maximal degree d. We further show that when dtanh β < 1, with high probability over the Erdős–Renyi random graph G(n,d n), it holds that the mixing time of Gibbs samplers is n1+Θ(1 loglogn). Both results are tight, as it is known that the mixing time for random regular and Erdős–Renyi random graphs is, with high probability, exponential in n when (d−1) tanh β> 1, and dtanh β>1, respectively. To our knowledge our results give the first tight sufficient conditions for rapid mixing of spin systems on general graphs. Moreover, our results are the first rigorous results establishing exact thresholds for dynamics on random graphs in terms of spatial thresholds on trees. We introduce a method for the theoretical analysis of exponential random graph models. The method is based on a large-deviations approximation to the normalizing constant shown to be consistent using theory developed by Chatterjee and Varadhan [European J. Combin. 32 (2011) 1000-1017]. The theory explains a host of difficulties encountered by applied workers: many distinct models have essentially the same MLE, rendering the problems practically'' ill-posed. We give the first rigorous proofs of degeneracy'' observed in these models. Here, almost all graphs have essentially no edges or are essentially complete. We supplement recent work of Bhamidi, Bresler and Sly [2008 IEEE 49th Annual IEEE Symposium on Foundations of Computer Science (FOCS) (2008) 803-812 IEEE] showing that for many models, the extra sufficient statistics are useless: most realizations look like the results of a simple Erd o s-R ' e nyi model. We also find classes of models where the limiting graphs differ from Erd o s-R ' e nyi graphs. A limitation of our approach, inherited from the limitation of graph limit theory, is that it works only for dense graphs. We study the computational complexity of several natural problems arising in statistical physics and combinatorics. In particular, we consider the following problems: the mean magnetization and mean energy of the Ising model (both the ferromagnetic and the anti-ferromagnetic settings), the average size of an independent set in the hard core model, and the average size of a matching in the monomer-dimer model. We prove that for all non-trivial values of the underlying model parameters, exactly computing these averages is #P-hard. In contrast to previous results of Sinclair and Srivastava (2013) for the mean magnetization of the ferromagnetic Ising model, our approach does not use any Lee-Yang type theorems about the complex zeros of partition functions. Indeed, it was due to the lack of suitable Lee-Yang theorems for models such as the anti-ferromagnetic Ising model that some of the problems we study here were left open by Sinclair and Srivastava. In this paper, we instead use some relatively simple and well-known ideas from the theory of automatic symbolic integration to complete our hardness reductions.