2/4 A Brief Introduction to Mean Field Spin Glass Models
CC Attribution - NonCommercial - NoDerivatives 4.0 International:
You are free to use, copy, distribute and transmit the work or content in unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Jagannath, Aukosh
Historically, mean field spin glass models come from the study of statistical physics and have served as prototypical examples of complex energy landscapes. To tackle these questions statistical physicists developed a new class of tools, such as the cavity method and the replica symmetry breaking. Since their introduction, these methods have been applied to a wide variety of problems from statistical physics, to combinatorics, to data science. This course will serve as a high-level introduction to the basics of mean field spin glasses and is intended to introduce the students to the basic notions that will arise in other courses during the Seminaire. On the first day, we plan to cover the random energy model, the ultrametric decomposition of Gibbs measures in p-spin glass models and the connection to Poisson-Dirichlet statistics. On the second day, if there is time, we will also introduce notions of free energy barriers and overlap gaps and their connection to spectral gap inequalities and algorithmic hardness results. Suggested Prerequisites: - Measure theoretic probability; - Point processes and their definition as random probability measures; - Basic notions from Gaussian analysis (concentration of measure, Slepian's interpolation inequality)
 Alright, so... Okay. Wha... Okay, so, um, all right, so, so just to sort of recap, where, where, where are we before the break? Hey. Sorry, one second. There seems to be some sort of echo. Does anyone else hear that? Yeah, I do. Hopefully it's over now. Hopefully it's, hopefully it's gone. Um, all right, so, yeah, so just to recap, you know, you know, we studied the random energy model, but at least we started, you know, to get a hint or at least see sort of a simple version of this notion of replica symmetry breaking. And so, um, so what I like to do in this part of the talk is to sort of explain sort of more broadly the replica symmetry breaking system and sort of introduce the canonical models to which, you know, the story for today and sort of tomorrow's lectures will be about. And so, okay. So what we'll do is we'll start. Let me just start by defining sort of the canonical models in this setting. So okay, so what we'll be thinking about for, you know, this part of today's lecture will be a mean field spin system on n particles. So again, you know, as before the configuration space for the system will be the discrete hypercube and on the discrete hypercube will, will put a few, you know, different kinds of Hamiltonians sort of the first one we want to talk about is we'd mentioned this already was the Sherrington-Cripatrick model or SK model. And here, what we'll do is we'll have it's an infinite range two-body interaction. So here we normalize things so that, you know, Jij are going to be standard Gaussian random variables iid and zero one. And this normalization is, you know, one thing you should check is that the variance here will be of order n. You know, the idea being that, you know, each particle will sort of contribute an order one quantity and the energy itself sort of maximally will be of order the number of particles in the system. That's sort of why we chose this normalization. So the next class is what's called Derrida's p-spin models. So here instead of having an infinite range two-body interaction will have an infinite range p-body interaction. And then, you know, you could say, okay, well, why just stop at sort of two-body or three-body or five-body why not have, you know, mixtures of these kinds of interactions. And so you can think about mixed p-spin glasses which are just linear combinations of these p-spin models. Okay. So the story I'll sort of talk about today is supposed to apply to sort of this broad class of models. And we at least know it for sort of a large class of mixed p-spin models now. And sort of fortunately, you know, it's a large enough class that when it comes to sort of doing useful calculations when we go to sort of applications and other courses to other fields such as, you know, combinatorics, etc. You can sort of approximate any model you want by the class we can consider, you know, in a way that sort of doesn't affect any of the main calculations you're going to try to sort of shoot for. And if what I said seems extremely vague, it was sort of deliberately so. But feel free to ask more about that during the break. So, you know, the questions we want to think about again is, so can we try to understand the structure of the Gibbs measure, you know, whatever that means in a mean field setting? And specifically, you know, we want to understand it in this sort of low temperature replica symmetry breaking phase. And then after this, we'll talk a bit about the free energy for these systems. And we'll sort of end by discussing the phase diagram and sort of what's known about what systems are replica symmetry breaking and when they are, as you vary the temperature, and for that last part of the discussion, it will also be helpful to add an external field by which I mean we'll consider the sort of modified Hamiltonian, which is say the piece spin model plus n times an external field times the magnetization, sorry, which so first up is to try to sort of understand the structure of the Gibbs measure in the replica symmetry breaking phase. And so what I'd like to do now is start to sort of explain what the predicted and, you know, now known in a broad class of models structure of the Gibbs measure and sort of the idea behind this was introduced in a bunch of papers by Mazzard, Parisi and Vera Sorro and various colleagues as now sometimes called the Mazzard, Parisi, Vera Sorro picture or onsets for the Gibbs measure. And so their idea is that in the replica symmetry breaking phase, the Gibbs measure should decompose into what are called pure states. So okay, so that, you know, for those of you who are familiar with sort of other statistical mechanics models sort of non-dissorted systems, you know, it's sort of similar in spirit to this idea of pure states you see there but sort of very different in terms of how it plays out. And in particular, you know, one of the really, so first you have to sort of understand what you mean by pure states in the setting, you know, the standard sort of DLR approach to Gibbs measures won't work because you're studying a mean field setup. But sort of the really incredible thing is not just that it decomposes into pure states, but that these pure states, if you look at the sort of space of pure states, they say it should have this really incredible decomposition. And what they say happens is, well, okay, so if I draw my space of pure states, which is going to be a ball in infinite dimensions. So this is my sort of rough attempt at the space of pure states. In space, what will happen is you'll have your pure states. There'll be tons of pure states at low temperature. But these pure states are themselves going to organize into groups. In the following sense, you know, every point, so every distinct point in these groups, they're going to be equidistant from each other. And these groups themselves are going to decompose into groups. Where again, every point in the green groups is going to be sort of equidistant from the points in the other green groups. And in particular, you know, there's sort of hiding behind all of this is the sort of incredible branching structure. What I'm saying is, you know, at the highest level, the green groups, all the states in the green groups are going to be orthogonal to each other, I mean, are going to sort of be equidistant to each other within these groups. Sort of the Berry centers, which are going to be themselves, you know, equally separated. And then within these groups, you have a separation as well. So we expect is this sort of really incredible sort of clustering of states. You know, I draw this picture where the idea is, you know, every point, each of these leaves that are connected by a single parent will be equidistant, etc., etc. So what we expect is that, so this example is what's called three RSP. So there's three levels of replica symmetry breaking here. And you see what's happening is that you have clusters of states, which are within clusters of states, which are themselves within clusters of states. The number of levels of clustering corresponds to the number of levels of replica symmetry breaking. You know, the idea being, you know, if I, okay, let me just continue. And furthermore, you know, there's, you know, the picture is even more precise, you know, if I fix to a certain depth in this tree, and I marginalize over all the states at that depth, and I look at the fractional mass each step takes, what you find is that it fixed depth. The rank weights follow a Poisson-Dierre-Claire structure. And more precisely, they follow what's sometimes called the Derrida Royal Probability Cascade. And the idea of, you know, an RPC is, you know, just, you know, just as in the REM case, where you just had this point process of the largest weights, and then the true Gibbs measure was just the math was you just sort of renormalized by the weights. For the Royal Cascade, sort of very vaguely, the idea is you're going to assign independent Poisson point processes to each of these vertices in such a way that the masses at any given level will be a Poisson-Dierre-Claire process. And sort of, you know, as if this weren't crazy enough, one of the central predictions is that you can have a continuum of levels of clustering. And this is sometimes called full replica symmetry breaking. And then finally, they predict that sort of at the finest level, you know, under a certain assumption, which I'll just call the EA property, the states at the finest level are pure in the sense that sort of if you just look at that state, it's a replica symmetric state. Okay. So that's sort of a lot to digest. And so what I'm doing now for the next chunk of time is to sort of start to unravel, you know, this picture and what we know about. So the starting point is, you know, I want to try to understand, you know, what does this sort of crazy branching structure have to do with replica symmetry breaking? At least, you know, as we've started to understand it in the last lecture. But remember last time around, we said, okay, replica symmetry breaking should relate to sort of the behavior of independent copies of the Gibbs measure. And so what we're going to want to do now is let's look, you know, not just at two copies of the Gibbs, of from the Gibbs measure. But let's draw infinitely many IID copies. So Marius is asking in the chat if you could define what state means in this context of the original models that you defined. So we're going to get to that. That by itself is a very non-trivial answer, I'd say. So hopefully I'll get to this in like a few minutes. And Gurev was asking for which models this picture is predicted to apply? All models is predicted to apply. But I mean, actually getting exact results, I think, is part of the difficult story here for mathematicians. It's supposed to hold for all of these sort of canonical models that we introduced. These mixed piece spin models, the piece spin models, and the SK model. Okay. So, okay, so how do we go from this sort of crazy clustering picture to something about replica? So we had this picture where we had states, whatever, you know, following the question really or whatever that means. And then within these states, you have states. So what does this have to do with replica symmetry breaking? How can I turn this into a question about the replica? Well, what I'll do is one thing we could think about is I'll form the following matrix, Rn, of normalized inner products between the different replicas. So it's a doubly infinite array. And what you expect in this setting is, you know, after applying a permutation, the draws you get should be organized in such a way that when you look at the Gram matrix, the Gram matrix has the following very structured form. Where sort of on the largest off diagonal, you have one allowed value of overlap, which you can think of as zero. Within the sort of red regime, you'll have another allowed value of overlap. And this allowed value will happen sort of in all of these regions here marked in red. And then at the finest level, you know, except for the diagonal, of course, you'll have yet another allowed value. Right so, you know, a replica symmetric picture is supposed to correspond to there just only being a q zero. So this infinite by infinite matrix you get is just on the off diagonals, it's just one number. And so you just have this huge matrix where, you know, in some such every entries, you can swap with any other entry. But when you have one level of replica symmetry breaking as we had in the random energy model, now you'll have two allowed values. You'll have, say, zero in the picture we have here. But it could be non zero if you have an external field. And then you'll have some other value of inner product. And then you could have, you know, in this picture here, we have two steps of replica symmetry breaking. We're going to sort of have three different allowed values in products. And again, the point being is that whenever you're within any one of these boxes, you have sort of the matrix should sort of look locally constant. All right. So okay, so what's going on here? So you know, behind all of this is the phenomenon of altimeter. So what do I mean by altimeter? If you have some metric space, it's called ultrometric. So for any sort of three points, x, y, and z, the distance between x and y satisfies the ultrometric triangle inequality. That's the maximum of the distance from x to z and the distance from z to y. This is to be contrasted with the usual triangle inequality where, you know, the distance is at most the sum. Right here, we're saying the distance is at most the maximum. So the simplest example of an ultrometric space is a tree. Whenever you think ultrometric, you should, you know, in the back of your mind, think about, think there's a tree sum. And more precisely, you know, if you let x denote the leaves of this tree, and on it, you impose the metric, which is just, so this is actually, sorry, a rooted tree. And the distance between x and y, what we'll do is we'll say it's the depth of the least common ancestor of x and y. And this will be my metric, maybe subtracted from this by the actual size of the tree. And so the exercise, so in the homework, you'll sort of go over an exact, a precise example of how you find an ultrometric space in a tree. But the sort of key points I want to bring up are the following properties of ultrometric spaces. So one thing to keep in mind is that an ultrometric space, all triangles are acute and isosceles. So what do I mean by this? You know, all the triangles, you have two sides, which are equal. But the two sides that are equal are also the longest sides. The next thing, sort of useful to note is that if I have two balls of say, radii r1 and r2, where r2 is bigger than r1, then the intersection, there's a dichotomy. It's either the empty set or it's the entire sort of smaller ball. Furthermore, if I have two points, x1 and a ball around x, and x2 and a ball around y, and if we know our priority that these are disjoint, then the distance between these two points is actually equal to the distance between the centers. And finally, this is sort of the last thing, the reason why I say when you think ultrometric space, you should think tree. Well think what you get is this. If you fix the radius, by property 2, if I fix the radius, the balls of that radius form a partition of the space. They're disjoint and they'll exhaust the full space. But it also tells you as you make smaller and smaller radii, these new balls aren't going to sort of lie willy nilly, they're going to be nested within the bigger balls. So in particular, if you have say a finite sequence of radii, then there's a family of partitioned, partition indexed by a depth k tree, where the family of balls form a partition, which is a filtration. In the sense that what you can do is, if I have, the picture is going to be, first let me take my largest balls, and then we'll say, okay, within those largest balls, there'll be maybe some smaller balls. And then what I'm saying is that these two balls, and this ball, if you can group them in the following way, you index to this level the red balls, and then you label the vertices at this level with the green balls. And the point is that this labeling is natural in the sense that if alpha is a child of base, beta, then the ball corresponding to alpha is contained in the ball corresponding to beta. So where are we going to see ultrametricity in our spin glass model? What we'll do is the following. As before, we'll look at this doubly infinite matrix of pairwise inner products. Well if you think about it for a second, you'll realize that this sequence lies in a compact product space, and so in particular, it's tight. So it'll have weak limits. And looking at this weak limit, what we find is the following sort of important result of Pinchanko, who showed that for a broad class of spin glass models, which I'll call generic, the probability, I'll put this in quotes, that the support of the limiting Gibbs measure, the probability that that support is an ultrametric set, when endowed with the sort of inherited topology from little l2, an inherited metric, this probability is one. Or more precisely, one thing to observe is that this matrix is weakly exchangeable in the sense that if I relabel all of my copies, all of my IID copies, it won't change the law of this matrix. And so then one way of writing this is just saying the probability that the inner product between the first two guys is bigger than the minimum of the inner product between the first guy and the third draw, and the second guy in this third draw, the probability that this inequality holds is one. And to translate from the ultrametric inequality to this, just realize that I've switched from distances to inner products. And so means become maxes and inequality split. So just a word, since there are some questions about this, what do I mean by generic models? So generic models are a class of mixed p-spin models where, so it's a mixed p-spin models, so it's a linear combination of these p-spin models. And the assumption is that if I sum over those p, such that the coefficient corresponding to that p is nonzero, the inverse of p, this should be in there. And the interest in generic models, as we'll sort of talk about in the next slide, is that while it is in principle a subset of the spin glass models, so in particular this doesn't apply immediately to, say, the SK model or the p-spin models, what you can say, for example, is by a continuity argument, the free energy of any mixed p-spin model, so for example the p-spin models and the SK model, can be approximated arbitrarily well by a generic models. So if I want to compute the free energy model of, say, the SK model, what I'm saying is that there's this sequence of generic models whose free energy is converged to the true free energy, sort of after the limit when it goes to infinity. So the reason why this is helpful is, you know, at the end of the day, the sort of the main sort of reason for introducing the replica symmetry breaking line thoughts in the physics literature was that it allowed people to start making computations about free energies, large deviation rates functions, but sort of hiding behind all those calculations was some form of ultrametricity. And so what this theorem is telling us is that, you know, we get this ultrametricity for at least a dense class of models in the space of models. So okay, so just getting sort of very briefly to the other question that was sort of brought up, what do we mean? So okay, so now we see ultrametricity, what do we mean now by states and pure states? Right, so the idea, the question we want to ask is, you know, how do we define what these states are from the Gibbs measure at finite n? And the idea is as follows. So the starting point is to show that for generic models, if you look at the sequence of Gibbs measures, the sequence is what we'll call approximately ultrametric. So a precise definition of approximately ultrametric will take, you know, I think quite a bit of time away from our discussion, so I'm happy to speak about it in the breakout sessions, but I just want to explain what it means heuristically, because that's sort of the key point. The idea is that instead of looking at the Gibbs measure, instead of trying to pick out sets, you want to look at sort of the effective geometry of the hypercube as sort of viewed from the Gibbs measure. And the idea is that with probability in the law of the disorder, which I'll denote by bold P, with P probability tending to one, there's a collection of, there's a collection of point of sets of clusters indexed by this AR, by which I mean it's the infinite tree of depth R such that each non-root vertex has countably many children. Now these sets are almost balls, they're almost balls with a certain collection of radii, and these balls, for these balls, you have these sort of the properties one through four of ultrometric spaces with high probability in G. By which I mean, so for example, if I think about the pairwise distance property, what I'm saying is if I take two independent copies that lie in the same cluster, the chance of that they're much further than you expect is very small. But if I pick two points in two different clusters, the chance that they're much closer to each other than you expect is also very small. So the point is that these sets, sort of from a geometric measure theoretic sense, are sort of acting like balls in ultrometric spaces, and then the states are then just the Gibbs measure conditioned on these balls. All right, so I think maybe perhaps now is a good time to take a five minute break, or a short break. Up next is just, we'll talk about sort of answering one of the other questions about when do you see what kinds of replica symmetry breaking in what models. Yeah, so if anyone has any questions in the chat, please type in their own number. Okay. It seems like Reza answered a lot of your questions. Okay. So in that case, maybe we sort of continual on a little bit. I think that's a good idea. We'll end maybe early. Sort of leave more time for questions. So, so the last thing I want to do is I want to talk a bit about the free energy. And then go from there to talk about what we know about the phase diagram for spin glass models. And so the starting point is what's what's called the per easy formula for the free energy, which is you know predicted originally by per easy in the late 70s. And then we have a sort of a improved. And sort of a sequence of works by Guara. And telegrant. For even models for models with only even components and then finally by a penchenco. And then we have a building off of work of Eisenman sins and star. Which I want to say is around 2007. What it says is that the free energy of, you know, any mixed piece spin model converges almost surely to a number which is given by minimizing a certain volume over the space of probability measures on the unit interval. And the functional that you're trying to minimize is now called the preasy functional. And so what it looks like is as follows. You take your measure. And what you do is you're going to solve two PDE's. Let me let me describe the picture with an external field. Just since we're going to want to talk about face diagrams in a bit. So you solve two PDE's. And you evaluate them both at a certain point in space and time. Now these PDE's so you knew. They solve a certain family of Hamilton Jacobi Bellman equations. So they solve equations of the form DT phi. Times, you know, the Laplacian in space on phi plus another coefficient, which is the CDF of that measure mu times the gradient squared. And the final time data you choose here is different for the two problems and same with. So here, you know, this code this choice of coefficient will depend on the model you choose. And so for example, in the case of P spin models. It's like P times P minus one times T to the P minus two. And the choice of F depends on the geometry of the problem so so for us for since we're looking at the spin glasses on the hypercube for you. So this is the the log of the hyperbolic cosine of X. And for V. It'll just be. You know formula relate to replica symmetry breaking well this the idea is as follows what we do is we look at the sort of the unique minimizer of P. We'll call it mu star, which is called. We'll call it the per easy measure proving the uniqueness is of course yet another challenge. It was an important result of offing her and Chen from 2000 14. And the idea is that the preasy measure is, you know, this minimizer is essentially going to be the limit of the overlap distribution. And where this is actually inequality in the case of generic models. And so then the point is, we'll say that a model is replica symmetric. If the minimizer is just a direct mass at a certain point. And the replica symmetry breaking means that the support contains more than one. So what happens up happening in the various models are as follows so let's describe the phase diagram. So first let's let's talk a bit about the sharing to and Patrick model. So here what happens is as follows. So on the x axis I'll be drawing, you know, the inverse temperature on the y axis I'm doing the external field. And there's a certain curve. Which is called the deal made a towerless line. And sort of what was originally conjectured is that the model should be replica symmetric here. And replica symmetry breaking here and in fact it should be full replica symmetry breaking here so you should have exponentially many. I mean, so you should have a continuum of levels of replica symmetry breaking. So the state of the art, as we know it right now is sort of there's an important result of Fabio tonnalli. And we showed that below the at line you are replica symmetry breaking. And more recently there's some work of mine with Ian Tabasco. And we showed that outside of an exceptional set that's compact and away from the critical fields. You have replica symmetry breaking above the at line. And full replica symmetry breaking is still, you know, a really exciting open question. And very recently there's been important progress by a finger Chen and shown who showed that you have full replica symmetry breaking at zero temperature. And the point is that you don't have finite finite replica symmetry breaking. You have at least infinite. All right, so but what happens for the peace been models. So here the story is a bit more complicated. So we'll sort of serve as a bridge for our lecture tomorrow, where we start talking about dynamics and algorithms. What's expected to happen is the following picture. So, so one thing you could do is you could plot the analog of the deal made a Taurus line. You know what we can show now is the analog of the deal made a Taurus line looks like this. And so in particular, it's not the correct phase boundary it's sufficiently low temperature. Instead what's supposed to happen is that there's some other phase boundary, above which your replica symmetric. There's another phase boundary where you're one step of replica symmetry breaking. And then finally when you cross this phase boundary your full replica symmetry breaking. And so what's known at the moment is that again below the at line. You have replica symmetry breaking. So we have a very high external fields. We know that the deal made a Taurus line is eventually the correct phase boundary. But what's going on sort of in this entire region here is still you know from a mathematical perspective, a real mystery. And we'll talk a little bit about it tomorrow when we start getting to questions related to dynamics. This is maybe a good point to stop for today. All right. Sorry, so many issues. There are some questions for you in the chat. So the pink line in the dairy dot piece been model was the the analog of the deal made a Taurus line for for the piece been models. So, okay, the question they're getting. So a lot of questions, let me try to answer them in order. So, okay, or maybe reverse order so is the conjecture picture on the right for any P greater than equal to three yeah. So this picture is supposed to hold for all P bigger than or equal to three. At least the results that are known are also known for all P greater than or equal to three. And sort of in general there's sort of a long discussion of a T lines and sort of their generalization for piece been models and when you know they're sort of the correct phase boundaries in this work of, you know, myself in Tabasco. So, in 2016, it's in PTR. Hopefully that one person asked at least for a pointer to briefly describe the at line beyond that, you know, I guess I'll just say this because it'll come up in tomorrow's lecture. So there is an explicit formula for the at line. The first thing is supposed to be the following. There is a certain in the replica symmetry breaking sort of picture, sort of replica trick story. There's a certain quantity that shows up. That depends on the points of the support of the Prezi measure and this quantity is called the replicon Eigen Valley. And the at line is the curve where this Eigen value is zero. And sort of a dynamical interpretation of it I'll sort of give a bit in tomorrow's lecture hopefully before we move on to another question. Let's just thank August for his two talks. You can all unmute yourselves and we'll have a king clap. So, after this, what we'll do is
 Alright, so...  Okay.  Wha...  Okay, so, um, all right, so, so just to sort of recap, where, where, where are we before  the break?  Hey.  Sorry, one second.  There seems to be some sort of echo.  Does anyone else hear that?  Yeah, I do.  Hopefully it's over now.  Hopefully it's, hopefully it's gone.  Um, all right, so, yeah, so just to recap, you know, you know, we studied the random  energy model, but at least we started, you know, to get a hint or at least see sort of  a simple version of this notion of replica symmetry breaking.  And so, um, so what I like to do in this part of the talk is to sort of explain sort of  more broadly the replica symmetry breaking system and sort of introduce the canonical  models to which, you know, the story for today and sort of tomorrow's lectures will be about.  And so, okay.  So what we'll do is we'll start.  Let me just start by defining sort of the canonical models in this setting.  So okay, so what we'll be thinking about for, you know, this part of today's lecture will  be a mean field spin system on n particles.  So again, you know, as before the configuration space for the system will be the discrete  hypercube and on the discrete hypercube will, will put a few, you know, different kinds  of Hamiltonians sort of the first one we want to talk about is we'd mentioned this already  was the Sherrington-Cripatrick model or SK model.  And here, what we'll do is we'll have it's an infinite range two-body interaction.  So here we normalize things so that, you know, Jij are going to be standard Gaussian random  variables iid and zero one.  And this normalization is, you know, one thing you should check is that the variance here  will be of order n.  You know, the idea being that, you know, each particle will sort of contribute an order  one quantity and the energy itself sort of maximally will be of order the number of particles  in the system.  That's sort of why we chose this normalization.  So the next class is what's called Derrida's p-spin models.  So here instead of having an infinite range two-body interaction will have an infinite  range p-body interaction.  And then, you know, you could say, okay, well, why just stop at sort of two-body or three-body  or five-body why not have, you know, mixtures of these kinds of interactions.  And so you can think about mixed p-spin glasses which are just linear combinations of these  p-spin models.  Okay.  So the story I'll sort of talk about today is supposed to apply to sort of this broad  class of models.  And we at least know it for sort of a large class of mixed p-spin models now.  And sort of fortunately, you know, it's a large enough class that when it comes to sort  of doing useful calculations when we go to sort of applications and other courses to  other fields such as, you know, combinatorics, etc.  You can sort of approximate any model you want by the class we can consider, you know,  in a way that sort of doesn't affect any of the main calculations you're going to try  to sort of shoot for.  And if what I said seems extremely vague, it was sort of deliberately so.  But feel free to ask more about that during the break.  So, you know, the questions we want to think about again is, so can we try to understand  the structure of the Gibbs measure, you know, whatever that means in a mean field setting?  And specifically, you know, we want to understand it in this sort of low temperature replica  symmetry breaking phase.  And then after this, we'll talk a bit about the free energy for these systems.  And we'll sort of end by discussing the phase diagram and sort of what's known about what  systems are replica symmetry breaking and when they are, as you vary the temperature,  and for that last part of the discussion, it will also be helpful to add an external  field by which I mean we'll consider the sort of modified Hamiltonian, which is say the  piece spin model plus n times an external field times the magnetization, sorry, which  so first up is to try to sort of understand the structure of the Gibbs measure in the  replica symmetry breaking phase.  And so what I'd like to do now is start to sort of explain what the predicted and, you  know, now known in a broad class of models structure of the Gibbs measure and sort of  the idea behind this was introduced in a bunch of papers by Mazzard, Parisi and Vera  Sorro and various colleagues as now sometimes called the Mazzard, Parisi, Vera Sorro picture  or onsets for the Gibbs measure.  And so their idea is that in the replica symmetry breaking phase, the Gibbs measure should decompose  into what are called pure states.  So okay, so that, you know, for those of you who are familiar with sort of other statistical  mechanics models sort of non-dissorted systems, you know, it's sort of similar in spirit to  this idea of pure states you see there but sort of very different in terms of how it  plays out.  And in particular, you know, one of the really, so first you have to sort of understand what  you mean by pure states in the setting, you know, the standard sort of DLR approach to  Gibbs measures won't work because you're studying a mean field setup.  But sort of the really incredible thing is not just that it decomposes into pure states,  but that these pure states, if you look at the sort of space of pure states, they say  it should have this really incredible decomposition.  And what they say happens is, well, okay, so if I draw my space of pure states, which  is going to be a ball in infinite dimensions.  So this is my sort of rough attempt at the space of pure states.  In space, what will happen is you'll have your pure states.  There'll be tons of pure states at low temperature.  But these pure states are themselves going to organize into groups.  In the following sense, you know, every point, so every distinct point in these groups, they're  going to be equidistant from each other.  And these groups themselves are going to decompose into groups.  Where again, every point in the green groups is going to be sort of equidistant from the  points in the other green groups.  And in particular, you know, there's sort of hiding behind all of this is the sort of  incredible branching structure.  What I'm saying is, you know, at the highest level, the green groups, all the states in  the green groups are going to be orthogonal to each other, I mean, are going to sort of  be equidistant to each other within these groups.  Sort of the Berry centers, which are going to be themselves, you know, equally separated.  And then within these groups, you have a separation as well.  So we expect is this sort of really incredible sort of clustering of states.  You know, I draw this picture where the idea is, you know, every point, each of these leaves  that are connected by a single parent will be equidistant, etc., etc.  So what we expect is that, so this example is what's called three RSP.  So there's three levels of replica symmetry breaking here.  And you see what's happening is that you have clusters of states, which are within clusters  of states, which are themselves within clusters of states.  The number of levels of clustering corresponds to the number of levels of replica symmetry  breaking.  You know, the idea being, you know, if I, okay, let me just continue.  And furthermore, you know, there's, you know, the picture is even more precise, you know,  if I fix to a certain depth in this tree, and I marginalize over all the states at that  depth, and I look at the fractional mass each step takes, what you find is that it fixed  depth.  The rank weights follow a Poisson-Dierre-Claire structure.  And more precisely, they follow what's sometimes called the Derrida Royal Probability Cascade.  And the idea of, you know, an RPC is, you know, just, you know, just as in the REM case,  where you just had this point process of the largest weights, and then the true Gibbs measure  was just the math was you just sort of renormalized by the weights.  For the Royal Cascade, sort of very vaguely, the idea is you're going to assign independent  Poisson point processes to each of these vertices in such a way that the masses at any given  level will be a Poisson-Dierre-Claire process.  And sort of, you know, as if this weren't crazy enough, one of the central predictions  is that you can have a continuum of levels of clustering.  And this is sometimes called full replica symmetry breaking.  And then finally, they predict that sort of at the finest level, you know, under a certain  assumption, which I'll just call the EA property, the states at the finest level are pure in  the sense that sort of if you just look at that state, it's a replica symmetric state.  Okay.  So that's sort of a lot to digest.  And so what I'm doing now for the next chunk of time is to sort of start to unravel, you  know, this picture and what we know about.  So the starting point is, you know, I want to try to understand, you know, what does  this sort of crazy branching structure have to do with replica symmetry breaking?  At least, you know, as we've started to understand it in the last lecture.  But remember last time around, we said, okay, replica symmetry breaking should relate to  sort of the behavior of independent copies of the Gibbs measure.  And so what we're going to want to do now is let's look, you know, not just at two copies  of the Gibbs, of from the Gibbs measure.  But let's draw infinitely many IID copies.  So Marius is asking in the chat if you could define what state means in this context of  the original models that you defined.  So we're going to get to that.  That by itself is a very non-trivial answer, I'd say.  So hopefully I'll get to this in like a few minutes.  And Gurev was asking for which models this picture is predicted to apply?  All models is predicted to apply.  But I mean, actually getting exact results, I think, is part of the difficult story here  for mathematicians.  It's supposed to hold for all of these sort of canonical models that we introduced.  These mixed piece spin models, the piece spin models, and the SK model.  Okay.  So, okay, so how do we go from this sort of crazy clustering picture to something about  replica?  So we had this picture where we had states, whatever, you know, following the question  really or whatever that means.  And then within these states, you have states.  So what does this have to do with replica symmetry breaking?  How can I turn this into a question about the replica?  Well, what I'll do is one thing we could think about is I'll form the following matrix, Rn,  of normalized inner products between the different replicas.  So it's a doubly infinite array.  And what you expect in this setting is, you know, after applying a permutation, the draws  you get should be organized in such a way that when you look at the Gram matrix, the  Gram matrix has the following very structured form.  Where sort of on the largest off diagonal, you have one allowed value of overlap, which  you can think of as zero.  Within the sort of red regime, you'll have another allowed value of overlap.  And this allowed value will happen sort of in all of these regions here marked in red.  And then at the finest level, you know, except for the diagonal, of course, you'll have yet  another allowed value.  Right so, you know, a replica symmetric picture is supposed to correspond to there just only  being a q zero.  So this infinite by infinite matrix you get is just on the off diagonals, it's just one  number.  And so you just have this huge matrix where, you know, in some such every entries, you  can swap with any other entry.  But when you have one level of replica symmetry breaking as we had in the random energy model,  now you'll have two allowed values.  You'll have, say, zero in the picture we have here.  But it could be non zero if you have an external field.  And then you'll have some other value of inner product.  And then you could have, you know, in this picture here, we have two steps of replica  symmetry breaking.  We're going to sort of have three different allowed values in products.  And again, the point being is that whenever you're within any one of these boxes, you  have sort of the matrix should sort of look locally constant.  All right.  So okay, so what's going on here?  So you know, behind all of this is the phenomenon of altimeter.  So what do I mean by altimeter?  If you have some metric space, it's called ultrometric.  So for any sort of three points, x, y, and z, the distance between x and y satisfies the  ultrometric triangle inequality.  That's the maximum of the distance from x to z and the distance from z to y.  This is to be contrasted with the usual triangle inequality where, you know, the distance is  at most the sum.  Right here, we're saying the distance is at most the maximum.  So the simplest example of an ultrometric space is a tree.  Whenever you think ultrometric, you should, you know, in the back of your mind, think  about, think there's a tree sum.  And more precisely, you know, if you let x denote the leaves of this tree, and on it,  you impose the metric, which is just, so this is actually, sorry, a rooted tree.  And the distance between x and y, what we'll do is we'll say it's the depth of the least  common ancestor of x and y.  And this will be my metric, maybe subtracted from this by the actual size of the tree.  And so the exercise, so in the homework, you'll sort of go over an exact, a precise example  of how you find an ultrometric space in a tree.  But the sort of key points I want to bring up are the following properties of ultrometric  spaces.  So one thing to keep in mind is that an ultrometric space, all triangles are acute and isosceles.  So what do I mean by this?  You know, all the triangles, you have two sides, which are equal.  But the two sides that are equal are also the longest sides.  The next thing, sort of useful to note is that if I have two balls of say, radii r1  and r2, where r2 is bigger than r1, then the intersection, there's a dichotomy.  It's either the empty set or it's the entire sort of smaller ball.  Furthermore, if I have two points, x1 and a ball around x, and x2 and a ball around y,  and if we know our priority that these are disjoint, then the distance between these  two points is actually equal to the distance between the centers.  And finally, this is sort of the last thing, the reason why I say when you think ultrometric  space, you should think tree.  Well think what you get is this.  If you fix the radius, by property 2, if I fix the radius, the balls of that radius  form a partition of the space.  They're disjoint and they'll exhaust the full space.  But it also tells you as you make smaller and smaller radii, these new balls aren't  going to sort of lie willy nilly, they're going to be nested within the bigger balls.  So in particular, if you have say a finite sequence of radii, then there's a family of  partitioned, partition indexed by a depth k tree, where the family of balls form a partition,  which is a filtration.  In the sense that what you can do is, if I have, the picture is going to be, first let  me take my largest balls, and then we'll say, okay, within those largest balls, there'll  be maybe some smaller balls.  And then what I'm saying is that these two balls, and this ball, if you can group them  in the following way, you index to this level the red balls, and then you label the vertices  at this level with the green balls.  And the point is that this labeling is natural in the sense that if alpha is a child of base,  beta, then the ball corresponding to alpha is contained in the ball corresponding to  beta.  So where are we going to see ultrametricity in our spin glass model?  What we'll do is the following.  As before, we'll look at this doubly infinite matrix of pairwise inner products.  Well if you think about it for a second, you'll realize that this sequence lies in a compact  product space, and so in particular, it's tight.  So it'll have weak limits.  And looking at this weak limit, what we find is the following sort of important result of  Pinchanko, who showed that for a broad class of spin glass models, which I'll call generic,  the probability, I'll put this in quotes, that the support of the limiting Gibbs measure,  the probability that that support is an ultrametric set, when endowed with the sort of inherited  topology from little l2, an inherited metric, this probability is one.  Or more precisely, one thing to observe is that this matrix is weakly exchangeable in  the sense that if I relabel all of my copies, all of my IID copies, it won't change the  law of this matrix.  And so then one way of writing this is just saying the probability that the inner product  between the first two guys is bigger than the minimum of the inner product between the  first guy and the third draw, and the second guy in this third draw, the probability that  this inequality holds is one.  And to translate from the ultrametric inequality to this, just realize that I've switched from  distances to inner products.  And so means become maxes and inequality split.  So just a word, since there are some questions about this, what do I mean by generic models?  So generic models are a class of mixed p-spin models where, so it's a mixed p-spin models,  so it's a linear combination of these p-spin models.  And the assumption is that if I sum over those p, such that the coefficient corresponding  to that p is nonzero, the inverse of p, this should be in there.  And the interest in generic models, as we'll sort of talk about in the next slide, is that  while it is in principle a subset of the spin glass models, so in particular this doesn't  apply immediately to, say, the SK model or the p-spin models, what you can say, for example,  is by a continuity argument, the free energy of any mixed p-spin model, so for example  the p-spin models and the SK model, can be approximated arbitrarily well by a generic  models.  So if I want to compute the free energy model of, say, the SK model, what I'm saying is  that there's this sequence of generic models whose free energy is converged to the true  free energy, sort of after the limit when it goes to infinity.  So the reason why this is helpful is, you know, at the end of the day, the sort of the  main sort of reason for introducing the replica symmetry breaking line thoughts in the physics  literature was that it allowed people to start making computations about free energies, large  deviation rates functions, but sort of hiding behind all those calculations was some form  of ultrametricity.  And so what this theorem is telling us is that, you know, we get this ultrametricity  for at least a dense class of models in the space of models.  So okay, so just getting sort of very briefly to the other question that was sort of brought  up, what do we mean?  So okay, so now we see ultrametricity, what do we mean now by states and pure states?  Right, so the idea, the question we want to ask is, you know, how do we  define what these states are from the Gibbs measure at finite n?  And the idea is as follows.  So the starting point is to show that for generic models, if you look at the sequence  of Gibbs measures, the sequence is what we'll call approximately ultrametric.  So a precise definition of approximately ultrametric will take, you know, I think quite a bit of  time away from our discussion, so I'm happy to speak about it in the breakout sessions,  but I just want to explain what it means heuristically, because that's sort of the key point.  The idea is that instead of looking at the Gibbs measure, instead of trying to pick  out sets, you want to look at sort of the effective geometry of the hypercube as sort  of viewed from the Gibbs measure.  And the idea is that with probability in the law of the disorder, which I'll denote by bold  P, with P probability tending to one, there's a collection of, there's a collection of point  of sets of clusters indexed by this AR, by which I mean it's the infinite tree of depth  R such that each non-root vertex has countably many children.  Now these sets are almost balls, they're almost balls with a certain collection of radii,  and these balls, for these balls, you have these sort of the properties one through four  of ultrometric spaces with high probability in G.  By which I mean, so for example, if I think about the pairwise distance property, what  I'm saying is if I take two independent copies that lie in the same cluster, the chance of  that they're much further than you expect is very small.  But if I pick two points in two different clusters, the chance that they're much closer  to each other than you expect is also very small.  So the point is that these sets, sort of from a geometric measure theoretic sense, are sort  of acting like balls in ultrometric spaces, and then the states are then just the Gibbs  measure conditioned on these balls.  All right, so I think maybe perhaps now is a good time to take a five minute break, or  a short break.  Up next is just, we'll talk about sort of answering one of the other questions about  when do you see what kinds of replica symmetry breaking in what models.  Yeah, so if anyone has any questions in the chat, please type in their own number.  Okay.  It seems like Reza answered a lot of your questions.  Okay.  So in that case,  maybe we sort of continual on a little bit.  I think that's a good idea.  We'll end maybe early.  Sort of leave more time for questions.  So, so the last thing I want to do is I want to talk a bit about the free energy.  And then go from there to talk about what we know about the phase diagram for spin glass models.  And so the starting point is what's what's called the per easy formula for the free energy,  which is you know predicted originally by per easy in the late 70s.  And then we have a sort of a improved.  And sort of a sequence of works by Guara.  And telegrant.  For even models for models with only even components and then finally by a penchenco.  And then we have a building off of work of Eisenman sins and star.  Which I want to say is around 2007.  What it says is that the free energy of, you know, any mixed piece spin model converges almost surely to a number which is given by minimizing a certain  volume over the space of probability measures on the unit interval.  And the functional that you're trying to minimize is now called the preasy functional.  And so what it looks like is as follows.  You take your measure.  And what you do is you're going to solve two PDE's.  Let me let me describe the picture with an external field.  Just since we're going to want to talk about face diagrams in a bit.  So you solve two PDE's.  And you evaluate them both at a certain point in space and time.  Now these PDE's so you knew.  They solve a certain family of Hamilton Jacobi Bellman equations. So they solve equations of the form DT phi.  Times, you know, the Laplacian in space on phi plus another coefficient, which is the CDF of that measure mu  times the gradient squared.  And the final time data you choose here is different for the two problems and same with.  So here, you know, this code this choice of coefficient will depend on the model you choose.  And so for example, in the case of P spin models.  It's like P times P minus one times T to the P minus two.  And the choice of F depends on the geometry of the problem so so for us for since we're looking at the spin glasses on the hypercube for you.  So this is the the log of the hyperbolic cosine of X. And for V.  It'll just be.  You know formula relate to replica symmetry breaking well this the idea is as follows what we do is we look at the sort of the unique minimizer  of P.  We'll call it mu star, which is called.  We'll call it the per easy measure proving the uniqueness is of course yet another challenge. It was an important result of offing her and Chen from 2000 14.  And the idea is that the preasy measure is, you know, this minimizer is essentially going to be the limit of the overlap distribution.  And where this is actually inequality in the case of generic models.  And so then the point is, we'll say that a model is replica symmetric.  If the minimizer is just a direct mass at a certain point.  And the replica symmetry breaking means that the support contains more than one.  So what happens up happening in the various models are as follows so let's describe the phase diagram.  So first let's let's talk a bit about the sharing to and Patrick model.  So here what happens is as follows.  So on the x axis I'll be drawing, you know, the inverse temperature on the y axis I'm doing the external field.  And there's a certain curve.  Which is called the deal made a towerless line.  And sort of what was originally conjectured is that the model should be replica symmetric here.  And replica symmetry breaking here and in fact it should be full replica symmetry breaking here so you should have exponentially many.  I mean, so you should have a continuum of levels of replica symmetry breaking.  So the state of the art, as we know it right now is sort of there's an important result of Fabio tonnalli.  And we showed that below the at line you are replica symmetry breaking.  And more recently there's some work of mine with Ian Tabasco.  And we showed that outside of an exceptional set that's compact and away from the critical fields.  You have replica symmetry breaking above the at line.  And full replica symmetry breaking is still, you know, a really exciting open question. And very recently there's been important progress by a finger Chen and shown who showed that you have full replica symmetry breaking at zero temperature.  And the point is that you don't have finite finite replica symmetry breaking. You have at least infinite.  All right, so but what happens for the peace been models.  So here the story is a bit more complicated.  So we'll sort of serve as a bridge for our lecture tomorrow, where we start talking about dynamics and algorithms.  What's expected to happen is the following picture.  So, so one thing you could do is you could plot the analog of the deal made a Taurus line.  You know what we can show now is the analog of the deal made a Taurus line looks like this.  And so in particular, it's not the correct phase boundary it's sufficiently low temperature.  Instead what's supposed to happen is that there's some other phase boundary, above which your replica symmetric.  There's another phase boundary where you're one step of replica symmetry breaking.  And then finally when you cross this phase boundary your full replica symmetry breaking.  And so what's known at the moment is that again below the at line.  You have replica symmetry breaking.  So we have a very high external fields.  We know that the deal made a Taurus line is eventually the correct phase boundary.  But what's going on sort of in this entire region here is still you know from a mathematical perspective, a real mystery.  And we'll talk a little bit about it tomorrow when we start getting to questions related to dynamics.  This is maybe a good point to stop for today.  All right.  Sorry, so many issues.  There are some questions for you in the chat.  So the pink line in the dairy dot piece been model was the the analog of the deal made a Taurus line for for the piece been models.  So, okay, the question they're getting.  So a lot of questions, let me try to answer them in order.  So, okay, or maybe reverse order so is the conjecture picture on the right for any P greater than equal to three yeah. So this picture is supposed to hold for all P bigger than or equal to three.  At least the results that are known are also known for all P greater than or equal to three.  And sort of in general there's sort of a long discussion of a T lines and sort of their generalization for piece been models and when you know they're sort of the correct phase boundaries in this work of, you know, myself in Tabasco.  So, in 2016, it's in PTR.  Hopefully that one person asked at least for a pointer to briefly describe the at line beyond that, you know, I guess I'll just say this because it'll come up in tomorrow's lecture.  So there is an explicit formula for the at line.  The first thing is supposed to be the following. There is a certain in the replica symmetry breaking sort of picture, sort of replica trick story. There's a certain quantity that shows up.  That depends on the points of the support of the Prezi measure and this quantity is called the replicon Eigen Valley.  And the at line is the curve where this Eigen value is zero.  And sort of a dynamical interpretation of it I'll sort of give a bit in tomorrow's lecture hopefully  before we move on to another question. Let's just thank August for his two talks. You can all unmute yourselves and we'll have a king clap.  So, after this, what we'll do is
202006291130-Jagannath_0_203.png 202006291130-Jagannath_204_3011.png 202006291130-Jagannath_3012_3059.png 202006291130-Jagannath_3060_3455.png 202006291130-Jagannath_3456_4259.png 202006291130-Jagannath_4260_5159.png 202006291130-Jagannath_5160_10643.png 202006291130-Jagannath_10644_10691.png 202006291130-Jagannath_10692_10739.png 202006291130-Jagannath_10740_10811.png 202006291130-Jagannath_10812_11459.png 202006291130-Jagannath_11460_11699.png 202006291130-Jagannath_11700_12215.png 202006291130-Jagannath_12216_12359.png 202006291130-Jagannath_12360_12467.png 202006291130-Jagannath_12468_13007.png 202006291130-Jagannath_13008_13787.png 202006291130-Jagannath_13788_13847.png 202006291130-Jagannath_13848_14171.png 202006291130-Jagannath_14172_14471.png 202006291130-Jagannath_14472_15059.png 202006291130-Jagannath_15060_15239.png 202006291130-Jagannath_15240_15299.png 202006291130-Jagannath_15300_15479.png 202006291130-Jagannath_15480_15527.png 202006291130-Jagannath_15528_15743.png 202006291130-Jagannath_15744_15791.png 202006291130-Jagannath_15792_15863.png 202006291130-Jagannath_15864_15923.png 202006291130-Jagannath_15924_15983.png 202006291130-Jagannath_15984_16043.png 202006291130-Jagannath_16044_16091.png 202006291130-Jagannath_16092_16139.png 202006291130-Jagannath_16140_16187.png 202006291130-Jagannath_16188_16235.png 202006291130-Jagannath_16236_16307.png 202006291130-Jagannath_16308_16367.png 202006291130-Jagannath_16368_16427.png 202006291130-Jagannath_16428_16487.png 202006291130-Jagannath_16488_16547.png 202006291130-Jagannath_16548_16727.png 202006291130-Jagannath_16728_16787.png 202006291130-Jagannath_16788_16859.png 202006291130-Jagannath_16860_17495.png 202006291130-Jagannath_17496_17759.png 202006291130-Jagannath_17760_21359.png 202006291130-Jagannath_21360_27419.png 202006291130-Jagannath_27420_27467.png 202006291130-Jagannath_27468_27515.png 202006291130-Jagannath_27516_27563.png 202006291130-Jagannath_27564_32147.png 202006291130-Jagannath_32148_36383.png 202006291130-Jagannath_36384_36491.png 202006291130-Jagannath_36492_48587.png 202006291130-Jagannath_48588_48635.png 202006291130-Jagannath_48636_49259.png 202006291130-Jagannath_49260_50159.png 202006291130-Jagannath_50160_52955.png 202006291130-Jagannath_52956_54347.png 202006291130-Jagannath_54348_56651.png 202006291130-Jagannath_56652_56819.png 202006291130-Jagannath_56820_57179.png 202006291130-Jagannath_57180_57359.png 202006291130-Jagannath_57360_57911.png 202006291130-Jagannath_57912_58163.png 202006291130-Jagannath_58164_58259.png 202006291130-Jagannath_58260_59159.png 202006291130-Jagannath_59160_60059.png 202006291130-Jagannath_60060_60959.png 202006291130-Jagannath_60960_61859.png 202006291130-Jagannath_61860_67307.png 202006291130-Jagannath_67308_67355.png 202006291130-Jagannath_67356_67403.png 202006291130-Jagannath_67404_76943.png 202006291130-Jagannath_76944_77003.png 202006291130-Jagannath_77004_84731.png 202006291130-Jagannath_84732_84779.png 202006291130-Jagannath_84780_85247.png 202006291130-Jagannath_85248_87263.png 202006291130-Jagannath_87264_87959.png 202006291130-Jagannath_87960_88859.png 202006291130-Jagannath_88860_89147.png 202006291130-Jagannath_89148_89243.png 202006291130-Jagannath_89244_89276.png