The best current estimates of the thresholds for the existence of solutions in random constraint satisfaction problems ('CSPs') mostly derive from the first and the second moment method. Yet apart from a very few exceptional cases these methods do not quite yield matching upper and lower bounds. According to deep but non-rigorous arguments from statistical mechanics, this discrepancy is due to a change in the geometry of the set of solutions called condensation that occurs shortly before the actual threshold for the existence of solutions (Krzakala, Montanari, Ricci-Tersenghi, Semerjian, Zdeborova: PNAS 2007). To cope with condensation, physicists have developed a sophisticated but non-rigorous formalism called Survey Propagation (Me-zard, Parisi, Zecchina: Science 2002). This formalism yields precise conjectures on the threshold values of many random CSPs. Here we develop a new Survey Propagation inspired second moment method for the random k-NAESAT problem, which is one of the standard benchmark problems in the theory of random CSPs. This new technique allows us to overcome the barrier posed by condensation rigorously. We prove that the threshold for the existence of solutions in random k-NAESAT is 2k-1ln2-(ln 2 2+1 4)+ek, where |ek| ≤ 2-(1-ok(1))k, thereby verifying the statistical mechanics conjecture for this problem.
Survey Propagation guided decimation. The SP formalism has given rise to an efficient message passing algorithm called ( SPD') @cite_9 . Experimentally, SPD seems spectacularly successful at solving, e.g., random @math -SAT for small values of @math . Unfortunately, no quantitative analysis of this algorithm is currently known (not even a non-rigorous one). The basic idea behind SPD is to approximate the marginals of the SP distribution (i.e., the probability that a given variable is true' in a solution drawn from the SP distribution) via a message passing heuristic. Then a variable @math is selected according to some rule and is assigned a value based on the (approximate) marginal. The entire procedure is repeated on the decimated'' problem instance where @math has been eliminated, until (hopefully) a solution is found.
@cite_9
1833343854 2290712804 2054743378 2061302090
Let Φ be a uniformly distributed random k-SAT formula with n variables and m clauses. Non-constructive arguments show that Φ is satisfiable for clause variable ratios m n ≤ rk 2k ln 2 with high probability (Achlioptas, Moore: SICOMP 2006; Achlioptas, Peres: J. AMS 2004). Yet no efficient algorithm is know to find a satisfying assignment for densities as low as m n rk · ln(k) k with a non-vanishing probability. In fact, the density m n rk · ln(k) k seems to form a barrier for a broad class of local search algorithms (Achlioptas, Coja-Oghlan: FOCS 2008). On the basis of deep but non-rigorous statistical mechanics considerations, a message passing algorithm called belief propagation guided decimation for solving random k-SAT has been forward (Mezard, Parisi, Zecchina: Science 2002; Braunstein, Mezard, Zecchina: RSA 2005). Experiments suggest that the algorithm might succeed for densities very close to rk for k = 3, 4, 5 (Kroc, Sabharwal, Selman: SAC 2009). Furnishing the first rigorous analysis of belief propagation guided decimation on random k-SAT, the present paper shows that the algorithm fails to find a satisfying assignment already for m n ≥ ρ · rk k, for a constant ρ > 0 independent of k. Let @math be a uniformly distributed random @math -SAT formula with @math variables and @math clauses. For clauses variables ratio @math the formula @math is satisfiable with high probability. However, no efficient algorithm is known to provably find a satisfying assignment beyond @math with a non-vanishing probability. Non-rigorous statistical mechanics work on @math -CNF led to the development of a new efficient "message passing algorithm" called [M ', Science 2002]. Experiments conducted for @math suggest that the algorithm finds satisfying assignments close to @math . However, in the present paper we prove that the basic version of Survey Propagation Guided Decimation fails to solve random @math -SAT formulas efficiently already for @math with @math almost a factor @math below @math . Decimation is a simple process for solving constraint satisfaction problems, by repeatedly fixing variable values and simplifying without reconsidering earlier decisions. We investigate different decimation strategies, contrasting those based on local, syntactic information from those based on message passing, such as statistical physics based Survey Propagation (SP) and the related and more well-known Belief Propagation (BP). Our results reveal that once we resolve convergence issues, BP itself can solve fairly hard random k-SAT formulas through decimation; the gap between BP and SP narrows down quickly as k increases. We also investigate observable differences between BP SP and other common CSP heuristics as decimation proceeds, exploring the hardness of the decimated formulas and identifying a somewhat unexpected feature of message passing heuristics, namely, unlike other heuristics for satisfiability, they avoid unit propagation as variables are fixed. We describe message-passing and decimation approaches for lossy source coding using low-density generator matrix (LDGM) codes. In particular, this paper addresses the problem of encoding a Bernoulli(1 2) source: for randomly generated LDGM codes with suitably irregular degree distributions, our methods yield performance very close to the rate distortion limit over a range of rates. Our approach is inspired by the survey propagation (SP) algorithm, originally developed by (2002) for solving random satisfiability problems. Previous work by (2005) shows how SP can be understood as belief propagation (BP) for an alternative representation of satisfiability problems. In analogy to this connection, our approach is to define a family of Markov random fields over generalized codewords, from which local message-passing rules can be derived in the standard way. The overall source encoding method is based on message-passing, setting a subset of bits to their preferred values (decimation), and reducing the code