We consider a random sparse graph with bounded average degree, in which a subset of vertices has higher connectivity than the background. In particular, the average degree inside this subset of vertices is larger than outside (but still bounded). Given a realization of such graph, we aim at identifying the hidden subset of vertices. This can be regarded as a model for the problem of finding a tightly knitted community in a social network, or a cluster in a relational dataset. In this paper we present two sets of contributions: (i) We use the cavity method from spin glass theory to derive an exact phase diagram for the reconstruction problem. In particular, as the difference in edge probability increases, the problem undergoes two phase transitions, a static phase transition and a dynamic one. (ii) We establish rigorous bounds on the dynamic phase transition and prove that, above a certain threshold, a local algorithm (belief propagation) correctly identify most of the hidden set. Below the same threshold no local algorithm can achieve this goal. However, in this regime the subset can be identified by exhaustive search. For small hidden sets and large average degree, the phase transition for local algorithms takes an intriguingly simple form. Local algorithms succeed with high probability for ( _ in - _ out > _ out e ) and fail for ( _ in - _ out < _ out e ) (with ( _ in ), ( _ out ) the average degrees inside and outside the community). We argue that spectral algorithms are also ineffective in the latter regime. It is an open problem whether any polynomial time algorithms might succeed for ( _ in - _ out < _ out e ).
Let us finally mention that the probability measure ) can be interpreted as the Boltzmann distribution for a system of @math particles on the graph @math , with fugacity @math , and interacting attractively (for @math ). Statistical mechanics analogies were previously exploited in @cite_42 @cite_11 . (See also @cite_22 for the general community detection problem.)
@cite_42 @cite_22 @cite_11
2002700608 2123942277 2000381089 2963994072 189596042 2562674776 2167064216 2156027049 2081489139 1998664146 2039765418 2122908915 2018379915 2095564275 1844244698 1989465898 1986646471 97131980 2004724832 1665214252
We study the asymptotic statistical behavior of the 2-dimensional periodic Lorentz gas with an infinite horizon. We consider a particle moving freely in the plane with elastic reflections from a periodic set of fixed convex scatterers. We assume that the initial position of the particle in the phase space is random with uniform distribution with respect to the Liouville measure of the periodic problem. We are interested in the asymptotic statistical behavior of the particle displacement in the plane as the timet goes to infinity. We assume that the particle horizon is infinite, which means that the length of free motion of the particle is unbounded. Then we show that under some natural assumptions on the free motion vector autocorrelation function, the limit distribution of the particle displacement in the plane is Gaussian, but the normalization factor is (t logt)1 2 and nott1 2 as in the classical case. We find the covariance matrix of the limit distribution. In classical kinetic models a particle free path distribution is exponential, but this is more likely to be an exception than a rule. In this paper we derive a generalized linear Boltzmann equation (GLBE) for a general free path distribution in the framework of Alt’s model. In the case that the free path distribution has at least first and second finite moments we construct an asymptotic solution to the initial value problem for the GLBE for small mean free paths. In the special case of the one-speed transport problem the asymptotic solution results in a diffusion approximation to the GLBE. In the present work, we consider the well-posedness and asymptotics of grazing collisions limit of the spatially inhomogeneous Boltzmann equation with Coulomb interaction. Under the screening hypothesis on the cross-section @math of the Boltzmann collision operator, that is, @math , where @math , we prove that there exists a common lifespan @math such that for any @math , the Boltzmann equation admits a unique nonnegative solution @math in the function space @math when the initial data belongs to the weighted Sobolev space @math with @math and @math . Moreover, it is proved that the solution @math is uniformly bounded with respect to the parameter @math in the above function spaces. As a consequence, by using the ... Many problems of interest in computer science and information theory can be phrased in terms of a probability distribution over discrete variables associated to the vertices of a large (but finite) sparse graph. In recent years, considerable progress has been achieved by viewing these distributions as Gibbs measures and applying to their study heuristic tools from statistical physics. We review this approach and provide some results towards a rigorous treatment of these problems. AMS 2000 subject classifications: Primary 60B10, 60G60, 82B20. We present a new learning algorithm for Boltzmann machines that contain many layers of hidden variables. Data-dependent expectations are estimated using a variational approximation that tends to focus on a single mode, and dataindependent expectations are approximated using persistent Markov chains. The use of two quite different techniques for estimating the two types of expectation that enter into the gradient of the log-likelihood makes it practical to learn Boltzmann machines with multiple hidden layers and millions of parameters. The learning can be made more efficient by using a layer-by-layer “pre-training” phase that allows variational inference to be initialized with a single bottomup pass. We present results on the MNIST and NORB datasets showing that deep Boltzmann machines learn good generative models and perform well on handwritten digit and visual object recognition tasks. We consider in this paper the problem of sampling a high-dimensional probability distribution @math having a density with respect to the Lebesgue measure on @math , known up to a normalization constant @math . Such problem naturally occurs for example in Bayesian inference and machine learning. Under the assumption that @math is continuously differentiable, @math is globally Lipschitz and @math is strongly convex, we obtain non-asymptotic bounds for the convergence to stationarity in Wasserstein distance of order @math and total variation distance of the sampling method based on the Euler discretization of the Langevin stochastic differential equation, for both constant and decreasing step sizes. The dependence on the dimension of the state space of these bounds is explicit. The convergence of an appropriately weighted empirical measure is also investigated and bounds for the mean square error and exponential deviation inequality are reported for functions which are measurable and bounded. An illustration to Bayesian inference for binary regression is presented to support our claims. Probability theory, like much of mathematics, is indebted to physics as a source of problems and intuition for solving these problems. Unfortunately, the level of abstraction of current mathematics often makes it difficult for anyone but an expert to appreciate this fact. In this work we will look at the interplay of physics and mathematics in terms of an example where the mathematics involved is at the college level. The example is the relation between elementary electric network theory and random walks. Central to the work will be Polya’s beautiful theorem that a random walker on an infinite street network in d-dimensional space is bound to return to the starting point when d = 2, but has a positive probability of escaping to infinity without returning to the starting point when d ≥ 3. Our goal will be to interpret this theorem as a statement about electric networks, and then to prove the theorem using techniques from classical electrical theory. The techniques referred to go back to Lord Rayleigh, who introduced them in connection with an investigation of musical instruments. The analog of Polya’s theorem in this connection is that wind instruments are possible in our three-dimensional world, but are not possible in Flatland (Abbott [1]). The connection between random walks and electric networks has been recognized for some time (see Kakutani [12], Kemeny, Snell, and The time-dependent Boltzmann equation, which describes the propagation of radiation from a point source in a random medium, is solved exactly in Fourier space. An explicit expression in real space is given in two and four dimensions. In three dimensions an accurate interpolation formula is found. The average intensity at a large distance @math from the source has two peaks, a ballistic peak at time @math and a diffusion peak at @math (with @math the velocity and @math the diffusion coefficient). We find that forward scattering adds a tail to the ballistic peak in two and three dimensions, @math and @math , respectively. Expressions in the literature do not contain this tail. Abstract This paper presents a derivation and initial study of a new generalized linear Boltzmann equation (GLBE), which describes particle transport for random statistically homogeneous systems in which the distribution function for chord lengths between scattering centers is non-exponential. Such problems have recently been proposed for the description of photon transport in atmospheric clouds; this paper is a first attempt to develop a Boltzmann-like equation for these and other related applications. We study the kinetics of ballistic annihilation, A+A→0, with continuous initial particle velocity distributions. The concentration and the rms velocity are found to decay as c∼t -α and υ rms ∼t -β , respectively, with the relation α+β=1 holding in any spatial dimension. A «mean-field» Boltzmann equation for the evolution of the velocity distribution predicts that a and β depend strongly on the initial condition, a behavior which is confirmed numerically in one and two dimensions We study properties of multiple random walks on a graph under various assumptions of interaction between the particles. To give precise results, we make the analysis for random regular graphs. The cover time of a random walk on a random @math -regular graph was studied in [C. Cooper and A. Frieze, SIAM J. Discrete Math., 18 (2005), pp. 728-740], where it was shown with high probability (whp) that for @math the cover time is asymptotic to @math , where @math . In this paper we prove the following (whp) results, arising from the study of multiple random walks on a random regular graph @math . For @math independent walks on @math , the cover time @math is asymptotic to @math , where @math is the cover time of a single walk. For most starting positions, the expected number of steps before any of the walks meet is @math . If the walks can communicate when meeting at a vertex, we show that, for most starting positions, the expected time for @math walks to broadcast a single piece of information to each other is asymptotic to @math as @math . We also establish properties of walks where there are two types of particles, predator and prey, or where particles interact when they meet at a vertex by coalescing or by annihilating each other. For example, the expected extinction time of @math explosive particles ( @math even) tends to @math as @math . The case of @math coalescing particles, where one particle is initially located at each vertex, corresponds to a voter model defined as follows: Initially each vertex has a distinct opinion, and at each step each vertex changes its opinion to that of a random neighbor. The expected time for a unique opinion to emerge is the same as the expected time for all the particles to coalesce, which is asymptotic to @math . Combining results from the predator-prey and multiple random walk models allows us to compare expected detection times of all prey in the following scenarios: Both the predator and the prey move randomly, the prey moves randomly and the predators stay fixed, and the predators move randomly and the prey stays fixed. In all cases, with @math predators and @math prey the expected detection time is @math , where @math is the @math th harmonic number. Statistical physics models with hard constraints, such as the discrete hard-core gas model (random independent sets in a graph), are inherently combinatorial and present the discrete mathematician with a relatively comfortable setting for the study of phase transition. In this paper we survey recent work (concentrating on joint work of the authors) in which hard-constraint systems are modeled by the space @math of homomorphisms from an infinite graph @math to a fixed finite constraint graph @math . These spaces become sufficiently tractable when @math is a regular tree (often called a Cayley tree or Bethe lattice) to permit characterization of the constraint graphs @math which admit multiple invariant Gibbs measures. Applications to a physics problem (multiple critical points for symmetry-breaking) and a combinatorics problem (random coloring), as well as some new combinatorial notions, will be presented. For the spatially homogeneous Boltzmann equation with cutoff hard potentials, it is shown that solutions remain bounded from above uniformly in time by a Maxwellian distribution, provided the initial data have a Maxwellian upper bound. The main technique is based on a comparison principle that uses a certain dissipative property of the linear Boltzmann equation. Implications of the technique to propagation of upper Maxwellian bounds in the spatially-inhomogeneous case are discussed. We discuss the relationship between the classical Lagrange theorem in mathematics and the quantum statistical mechanics and thermodynamics of an ideal gas of multispecies quasiparticles with mutual fractional exclusion statistics. First, we show that the thermodynamic potential and the density of the system are analytically expressed in terms of the language of generalized cluster expansions, where the cluster coefficients are determined from Wu’s functional relations for describing the distribution functions of mutual fractional exclusion statistics. Second, we generalize the classical Lagrange theorem for inverting the one complex variable functions to that for the multicomplex variable functions. Third, we explicitly obtain all the exact cluster coefficients by applying the generalized Lagrange theorem. @S0163-1829 98!03335-9# Abstract This paper extends a recently introduced theory describing particle transport for random statistically homogeneous systems in which the distribution function p ( s ) for chord lengths between scattering centers is non-exponential. Here, we relax the previous assumption that p ( s ) does not depend on the direction of flight Ω ; this leads to a new generalized linear Boltzmann equation that includes angular-dependent cross sections, and to a new generalized diffusion equation that accounts for anisotropic behavior resulting from the statistics of the system. Abstract This paper studies the approximation of the Boltzmann equation by the Landau equation in a regime when grazing collisions prevail. While all previous results in the subject were limited to the spatially homogeneous case, here we manage to cover the general, space-dependent situation, assuming only basic physical estimates of finite mass, energy, entropy and entropy production. The proofs are based on the recent results and methods introduced previously in [R. Alexandre, C. Villani, Comm. Pure Appl. Math. 55 (1) (2002) 30–70] by both authors, and the entropy production smoothing effects established in [R. , Arch. Rational Mech. Anal. 152 (4) (2000) 327–355]. We are able to treat realistic singularities of Coulomb type, and approximations of the Debye cut. However, our method only works for finite-time intervals, while the Landau equation is supposed to describe long-time corrections to the Vlasov–Poisson equation. If the mean-field interaction is neglected, then our results apply to physically relevant situations after a time rescaling. We use entropy ideas to study hard-core distributions on the independent sets of a finite, regular bipartite graph, specifically distributions according to which each independent set I is chosen with probability proportional to λ∣I∣ for some fixed λ > 0. Among the results obtained are rather precise bounds on occupation probabilities; a ‘phase transition’ statement for Hamming cubes; and an exact upper bound on the number of independent sets in an n-regular bipartite graph on a given number of vertices. This chapter presents results on the application of the concept of entropy to estimation of distribution algorithms (EDAs). Firstly, the Boltzmann mutual information curves are introduced. They are shown to contain a lot of information about the difficulty of the functions. Next, a design method of discrete benchmark functions is presented. The newly developed approach allows the construction of both single and random classes of functions that obey a given collection of probabilistic constraints. This application and the next — the construction of low cost search distributions — are based on the principle of maximum entropy. The last proposal is the linear entropic mutation (LEM), an approach that measures the amount of mutation applied to a variable as the increase of its entropy. We argue that LEM is a natural operator for EDAs because it mutates distributions instead of single individuals. Consider the problem of approximately counting weighted independent sets of a graph G with activity λ, i.e., where the weight of an independent set I is λ|I|. We present a novel analysis yielding a deterministic approximation scheme which runs in polynomial time for any graph of maximum degree Δ and λ< λc=(Δ-1)Δ-1 (Δ-2)Δ. This improves on the previously known general bound of λ ≤ (2 ‾ Δ-2). The new regime includes the interesting case of λ=1 (uniform weights) and Δ ≤ 5. The previous bound required Δ ≤ 4 for uniform approximate counting and there is evidence that for Δ ≥ 6 the problem is hard. Note that λc is the critical activity for uniqueness of the Gibbs measure on the regular tree of degree Δ, i.e., for λ ≤ λc the probability that the root is in the independent set is asymptotically independent of the configuration on the leaves far below. Indeed, our analysis is focused on establishing decay of correlations with distance in the above weighted distribution. We show that on any graph of maximum degree Δ correlations decay with distance at least as fast as they do on the regular tree of the same degree. This resolves an open conjecture in statistical physics. Our comparison of a general graph with the tree uses an algorithmic argument yielding the approximation scheme mentioned above. Also, by existing arguments, establishing decay of correlations for all graphs and λ<λc gives that the Glauber dynamics is rapidly mixing in this regime. However, the implication from decay of correlations to rapid mixing of the dynamics is only known to hold for graphs of subexponential growth, and hence, our result regarding the Glauber dynamics is limited to this class of graphs. Restricted Boltzmann machines were developed using binary stochastic hidden units. These can be generalized by replacing each binary unit by an infinite number of copies that all have the same weights but have progressively more negative biases. The learning and inference rules for these "Stepped Sigmoid Units" are unchanged. They can be approximated efficiently by noisy, rectified linear units. Compared with binary units, these units learn features that are better for object recognition on the NORB dataset and face verification on the Labeled Faces in the Wild dataset. Unlike binary units, rectified linear units preserve information about relative intensities as information travels through multiple layers of feature detectors.