Mean field games with major and minor players
CC Attribution - NonCommercial - NoDerivatives 2.0 Generic:
You are free to use, copy, distribute and transmit the work or content in unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Carmona, René
We introduce a new strategy for the solution of Mean Field Games in the presence of major and minor players. This approach is based on a formulation of the fixed point step in spaces of controls. We use it to highlight the differences between open and closed loop problems. We illustrate the implementation of this approach for linear quadratic and finite state space games, and we provide numerical results motivated by applications in biology and cyber-security.
 Over the years I've been giving a few talks and each time I thank the organizer. But this time I really mean I want to thank the organizer. A few years ago, maybe over 40, in 69, I passed what some of you may still know, aggregation, and with all the people from 68. And in July I had two options. I mean three options actually. Going to teach in a high school somewhere in the middle of the country. I was not excited. And then I had an offer from Grenoble and an offer from Lumini. Lumini was just being created. So it was just a no-man's land and there was this half of this huge building of nine floors. And people were pushing me, you know, Grenoble, you know, this is the high point of PDEs, Malgrange is there, you should go blah blah blah. And at that time I decided that now I'm not the kind of guy who is going to leave Marseille. I'm going to stay in Marseille for the rest of my life. Well, this afternoon I'll be happy 40 years later to go back to the Karang. All right, so I'm going to talk about a mean field game and I didn't know, you know, what would be covered in some of the tutorial lectures. So I'm going to talk about something a little bit off the main beaten path. And I'm going to talk about games with major and minor players. And then at the end I will talk about something that I wanted really bad to talk about, but for which I didn't have the time to actually do computation and prepare illustration for today. You know, at my age if you start learning a new programming language it takes more time than what you think. All right, so I'm going to talk about mean field game with major and minor player. And someone told me that this was working, but maybe not. Oh, defy any logic. This is a left-handed gadget, I guess, not a right-handed guy. All right, so this is the typical starting point if the models are models with the stochastic differential equation. The second part of the talk I'm going to talk about an application where the model will not be given by stochastic differential equation, but at least this will give us a little bit of a sense of what we're talking about. I'm just playing with this gadget, but it doesn't seem to be. Why do I go back? Here we go. All right, so we have a major player and its state will be denoted by, with the superscript zero and we have a generic player and there will be in large number, and this is why I call it generic, which state will be denoted by xt. And both states are evolving according to a stochastic differential equation, they're ETO processes, but what is important to notice is that the control of the major player, which will be denoted by alpha superscript zero, of course occurs in the dynamic of the state of the major player. This guy controls its own destiny, but it also appears in the dynamics of the minor players. And so this is in this sense that the major player influences the minor players. Also the state x0t of the major player appears in the dynamics of the state of the minor player, so if you think in terms of financial application, you look at the financial system, for example in the US, you have thousands and thousands of small banks, and then you have a small number, finite number of systemically important institution, think of Goldman Sachs, and whatever Goldman Sachs does is going to affect whatever all the other banks do. On the other hand, whatever the local bank in Irvine, California does, is not going to worry too much, Goldman. So the dynamics of the major player state is not affected by the individual state of the player, still you have this mu t entering in both stochastic differential equation, and this mu t represents the empirical distribution of the states of the minor players. So the major player feels the field of the minor players, it does not feel them individually, Goldman as I said doesn't care about what the savings and loan in Irvine do, but on the other hand, Goldman may worry about the proportion of savings and loans doing this or doing that. So that's how the system is mean field, it is mean field among the major, sorry, among the minor players, and this field of minor players is taken into account by the major player, but on the other hand, the major player state doesn't depend upon the individual minor players. So this creates this symmetry and we're going to assume that we have a large number, an infinite number, continuum if you want of minor players and just one minor player, a major player, and this major player could be multi-dimensional, in other words, you could have Goldman, Morgan, Stanley, JP Morgan, but only in finite numbers. Then each of these players, major or minor, incurs a certain cost and the cost as usual is given by an expectation of the effect of a running cost, an integral from zero to the terminal time capital T and a terminal cost, G0G, and again we have the same structure, namely the cost to the major player depends upon the field, upon the measure of UT of whatever the minor players do, but on the other hand, for the minor players, the cost running or terminal depends upon what exactly the state of the major player is, X0T, or even what actions the major player is taking. Wunderbar. All right, so I'm just going to give a few slides to propose a way to formulate the problems. By the way, most everything that I'm going to say except that the hint at the calculation which I couldn't finish is already in a beautiful book that Francois will sell you at a discount if you behave well during the summer school. So we're going to assume that the controls are first in open loop form. Open loop control are very convenient and probabilists love them. Practically I just believe they're nonsense, they're not practical, and in particular if you want to do a numeric calculation like Eva's been explaining to us this morning, you need to work with closed loop. However, there are many mathematical results that you can first prove for open loop formulation of the problem, and as I said, as probabilists, they are very convenient. So the notion of open loop can be formulated here by saying that the control alpha 0 will be a function of time and the entire path of the white noise of the state of the major player. The control of the minor player similarly will be a function of time and the entire path of the white noise of driving the equation of the major player and the white noise driving the equation of the minor players. Who sees these white noise in this path? I do not know. So practically if I wanted to construct this open loop control and take that as an action, I would have to see the entire path and in practice, as I say, unfortunately, we rarely see them. Sometimes we can, but it's very rare. And the function of these paths will be functions phi 0 and phi, which are deterministic functions defined on these spaces, the time, the path space, and taking value in sets A0 and A where the controls take value. All right, so when you formulate a major and minor model, you first look at the problem of the major player and you're going to see how does the major player going to respond to the field of minor player. So you first assume that all the minus player, which are behaving identically, choose an open loop control, so a function phi, which is going to be function of, as I say, time and the two white noise paths. And when all these minor players use this control, then the major player is facing the following problem, the equations for a state is the same as before, and then we have an equation now for the state of the minor player where the function phi comes in. These two equations are coupled because, of course, W0 appears in both of them. And now mu t is going to be taken to be the conditional law of xt given the path of the white noise of the major player. And so the major player problem is going to be to search for an optimal control, phi 0 star, this will be the optimum, minimizing is cost, minimizing is cost, where now the cost depends upon the function phi, which is chosen by all the minor players. But the points that I want to make here is that no matter what, the problem, the optimal control that the major player has to solve to find its best response to the decision, the behavior of this minor player is an optimal control of the McKean-Vlasov type because the distribution of the state here that is only the marginal of xt enters into the dynamic of the equation. So if you formulate the best response of the major player in this way, the problem you have here is an optimal control of McKean-Vlasov dynamics. Now we want to look at the best response of the minor player to the major player. So we're going to assume that the major player chooses a strategy of control alpha 0 given by feedback function phi 0, which is a function of the entire path. And we're going to assume that the field of minor players choose a feedback function phi, which is now a function of two white noses. And once this is done, the dynamics of the two states, x0 and xt, solve this stochastic differential equation, which is a stochastic differential equation, again, of the McKean-Vlasov type because the law of xt conditioned by the white noses w0 enters in this equation. But this is not a control problem. You solve this SDE, which is an SDE of the McKean-Vlasov type. But once you solve this SDE, you use the law of this solution and you inject that into the optimization that the minor player, the typical minor player should solve. And now you have an optimal control of the regular type. This is not of the McKean-Vlasov type. And once you can solve it as a function of phi0 and phi, we're going to denote by phi bar star the optimal open loop control, which is given by a function of the white noses path. Let's assume that this can be solved. Let's even assume that it is unique. And now finding a Nash equilibrium for a system of major and minor player is finding a fixed point to the best response map, namely finding a set of couple of feedback function phi0 and phi satisfying this equality. So I want to emphasize the fact that I want to find a Nash equilibrium for the whole system, including major player and minor player. This is going to be in contrast with a simple example I will give, I will show a slide in a minute. So this, if I can solve that, will give me an open loop Nash equilibrium for the system. What is the closed loop version of this problem? I didn't solve it. Of course, I just formulated the problem. But the formulation of major and minor models of mean field game has been all over the map. These models were introduced by Huang and then Kane and Noorian. And many people have written paper on it, but I think there is a major confusion in how this model should be set. Anyway, so now if you want to have a closed loop version of this problem, you do exactly the same thing where now the control for the major player and for the minor players are given by feedback function, but now they're functions of the trajectory of the simple path of the states. So that still may be unreasonable to imagine that I can keep track of the entire path of the state before I make my decision at time t, but at least it is more reasonable from a practical point of view. And now if you want to have a more COVID version, a version for which we can do actual computation and write PDEs, et cetera, et cetera, you will now take the feedback function phi 0 and phi to be only function of the state at time t. Now why do I bother so much to write all this annoying definition is the fact that in general, these form of the definition of an open loop problem or closed loop problems play a major role in game theory when you work with finite many players. If you work with finite many player, looking for open loop equilibrium will lead to some solution which can be different than closed loop Nash equilibria. Even if the open loop Nash equilibrium which you find is in closed loop form. So it's not enough to have an open loop Nash equilibrium in closed loop form to be able to say I got a closed loop Nash equilibrium. That's the case. However, and this is not really a theorem, but it can be proved in many cases and it's some sort of a folk theorem, when the size of the population grow and the number of players tends to infinity, and this is basically the situation we set ourselves up when we do mean field game, these differences disappear in the wash. And this is why very often when you study mean field game, you don't care so much about closed loop open loop because whether you tackle for one or the other, very often the solutions are the same. Unfortunately when you deal with major and minor players, that's not going to be the case because the limit n tends to infinity which is going to give a lot of averaging happening. You remember the idiosyncratic white noise attached to each of the players, they're going to force some sort of law of large number and you're going to have things going to random nest which is going to disappear. And so while the random nest disappear, the differences between the closed loop and open loop disappear as well. That's not the case here because the W0 and the X0 are going to remain. We have one major player or finite number of major player, we have a large number of minor players, they have their idiosyncratic noise. When the size of the minor population grows to infinity, you get your averaging but you never average out the noise of the major player and you saw that the measure that I was taking were conditional measure. So this averaging is not there and it is likely and we're going to see example that closed loop and open loop Nash equilibrium still differ, still in a mean field limit. So the best response being found in these classes of control, again finding a Nash equilibrium is finding a fixed point. So this is an example of a paper which was started years ago and which is still not finished but where it looks like there is a major, it looks like there are minor players but it is not a Nash equilibrium of this type that we would be looking for. When you do contract theory, you have a principle and for my friend Ronea, let's see the regulator of the European community, not the US because we didn't sign the Paris Accord, it's not for us, this sort of thing. So we have a regulator and the regulator basically talk to all the electric company, RWE, DF, etc., and tell them, okay, so this is what we're going to do. This is the contract that I give you. You're going to be rewarded for reducing your emission, you're going to be penalized if you screw up and you go over your cap, whatever the regulator decides. So this is the major player and then you have the mass of minor player, all the little electric operators, the little people like IDF, and so they're going to react to this regulation and they're going to choose a behavior which depends upon the regulation set in place by the regulator. So you have also a symmetry between a small number of player, the big player being the regulator and a large number of player, the electric operators, but now this is different. The Nash equilibrium will be a Nash equilibrium among all the electric operators, not involving the regulator. The Nash equilibrium will be among the minor players only. So that's a slightly different kind of problem than the one that I had before where before I won a Nash equilibrium among everybody. In other words, I won the major player not having any incentive to change his contract once the minor players have chosen what to do. All right, so when can we solve this mean field game? We do not have many large classes of problems. So I'm going to stick to two simple models for which we can actually analytically solve the problems and do some numerics. I'm going to first look at linear quadratic model and then I'm going to look at models for which the state space is finite, in which case we will not have a stochastic differential equation, but intuitively we can make sense of how things are working. So a linear quadratic model will come in this form. The, no, it's still not working. The x0, the state of the major player, will satisfy a linear equation and I assume that the volatility is constant. And what is entering in this linear equation is x bar t, which is the conditional expectation because it's linear the way the measure comes in will be only through its mean. And so the minor players influence the major player decision only through the mean. And their dynamics, the dx t, involves also their means, but they involve the dynamics of the minor player state involve x0 t, which is the state of the major player. And same thing for the cost except that now we take them quadratic, linear quadratic, linear dynamic quadratic cost. Okay, so you have the same type of equation. Again the mean x t bar of the states of the minor players enters the cost of the major player, but that's the only way the minor players will influence the dynamics and the cost of the major player. So the reason we can do that is that again, you know, we can write down the optimization problem that I mentioned before. You have linear dynamic quadratic cost, so this optimization problem reduced to big matrix Riccati equation. And so whether you do that through an FBSD or through another approach, you know, you end up with big matrix Riccati equation and hopefully you can solve them. And in this particular case we can actually solve them in the case of open loop directly and we find a solution. In the case of closed loop, things are a little bit more difficult. However, if we search for controls in a specific form because we know in advance that the controls are going to be linear in the states, so we look already for controls which are linear in the states and the mean. Now when we try to find that these controls satisfy the Nash equilibrium, we perturb that with general control. So it's really a Nash equilibrium that I will find, but I search for a Nash equilibrium in this form, but that's not a restricted notion of a Nash equilibrium. That's going to be a general Nash equilibrium. And then again, you know, we can formulate the optimization problem in the form of a large FBSD and of course they are fine and we solve them through solving a large matrix Riccati equation. However, in this particular case they're different. So this means that the open loop and the closed loop solutions are different. So that happens even for mean field game problem and that happened even for linear quadratic case. So as I said, one of the reasons why I went through these little song and dance was to emphasize the fact that that's an issue, which are the important one. I mentioned that already three or four times. So I'm going to give you just a very simple naive application. So let's imagine that you're following a beehive. You want to see how the bees move from one location to another. So how do they do that? So the queen is getting old. There is a new queen in the bunch. So they want to get rid of the old queens. So the queen leaves the group with a certain number of worker bees following the queen. They didn't notice that there was a young and more beautiful queen. So they follow the old queen. And so they go and sit on a branch on a tree and they wait. And then a few bees called the stricter bees, they go and they look around in the neighborhood. They try to find a new location for the hive. So once they think they found a new location, they come back to the tree where the old queen and the workers are waiting there. And they start dancing and dancing, trying to convince this group to follow them. And eventually, one of them is dancing so well that the group decides, OK, we're going to follow this stricter bee. And they follow the stricter bee to the new location. That's a mean field game problem, just kidding. I'm going to pretend that's a mean field game problem and that's a linear quadratic mean field game problem. That's a linear quadratic mean field game problem with a major and minor player. And we're going to see how it works. So we're going to model only the velocity because it's simpler if you model the velocity and the position. It's a little bit more complicated mathematically because you have hyperliptic diffusion instead of diffusion and things are not as simple. So we're going to denote by v0n the velocity of the stricter bee, the bee which is going to go to the final destination, which knows where it is and which wants to drag the old queen and the bunch to this new location. So that's going to be the velocity of the stricter bee. And the control is alpha 0 up to noise. The stricter bee controls its own velocity. We have n, we have n capital N bees with the queen. And so they control the velocity, alpha it, and they have their own idiosyncratic noise. What is the cost to the stricter bee? So you have three components to the cost. Then we take a linear combination of these three components. The first component is the square of the difference between the velocity of the stricter bee and the optimal velocity it would take to go from point A to point B. So in other words, this new t here is a deterministic function which if the bee used that as a velocity it would take from point A to point B. Then the second term is I want the velocity of the stricter bee to be not very different from the average velocity of the group because the stricter bee wants to pull the group to the new location. And finally, just a kinetic energy term, you want not to be too tired and exhausted. You want to end up where you want to go. So that's the cost of the stricter bee. Now what are the cost of the little worker bees? So they have again a cost with three components. The first component is that they want their velocity to be similar to the velocity of the stricter bee. So they want to follow the stricter bee. They decided to do that so they want to do it. So that's the first term. The second term is that they want to stay in a group. So they want their individual velocity not to be much different than the average velocity of the group. And finally again, they want to end up where they want to be. Namely, they do not want to get exhausted too quickly. So that's the finite player game. You write the limit as n10 to infinity as a mean field game. And this is one example of numerics that you would get. So in this particular case, you know, this is silly. There is no new location. I take the velocity to be circular. Okay, so this is the velocity in use of t. And so you see in black the trajectory of a stricter bee following this velocity. And then at a distance, remember, the position is not part of the model. So if they start from different places, there is a plot of 10 trajectories of 10 of these worker bees. And you know, they try to move around in circle, move around in circle. Again the position doesn't matter. But this is for a certain number, a certain set of coefficient, the relative value of the penalty. But if you start monkey around with these coefficients, you know, even though the velocity, the optimal velocity is the same, well, you know, the stricter bee is not even going to follow that. And the worker bees are not even going to follow it. Okay, so that's a typical example. And there was another slide where, you know, the velocity in you was linear, and then you could see also that the worker bees were following. So you're going to have to buy the book to find the other plot. And let me tell you how outrageous it is. I mean, France is not even as upset as I am. But you know, they want to sell this crap for $149 a volume. Who is going to buy it? You know, if you have, I mean, I know people have ways to do it. Just get a copy, you don't have to buy it because, you know, this is ridiculous. $149, so multiply by two, that's basically $300. It's just ridiculous. But you would have more pictures. All right. So that's what it is. So this plot is an illustration of what's really behind mean field game. Stefan, when do I have to close? You have to close it. Okay. Okay. All right. So this is an illustration of what's behind the theory of mean field game, which is based on a result you're going to hear about over and over and over during the week and presumably during the many weeks if you stick around is the propagation of chaos. Here it's a simple form of conditional propagation of chaos in the following sense. What I did here is I took five workabees, five minor players, okay, and in condition I generate one path for the white noise of the major player, namely the street cubby, okay. So this, what we're seeing here is conditioned, this is conditioned on the white noise of the major player, okay. But condition on that, I take my five street cubbies and I look only at a group of five street cubbies. I simulate their position and I compute the correlation matrix. So if you compute the correlation matrix, you see that you have, so it's a five by five matrix, and on the diagonal you have high terms, they should be one, and outside the diagonal it is still relatively high, but it's definitely non-zero. Now if you, if I redo the same thing, but my five minor players, I looked at them in a group of ten minor players, okay. So I redo the same simulation. I have now ten minor players, only one path of fixed path of the white noise of the major player, and I do the same thing and I compute the correlation, and it is much lighter outside the diagonal. And now if I take 20, and if I take 50, and if I take 100, you see that my correlation matrix is diagonal. Given the fact that we are linear quadratic, this is a Gaussian model, so this means that they're independent. So in other words, you know, when the bath, when the field of particle is larger and larger, a fixed number of particles behave exactly as if they were independent. Here it's conditioned on the white noise of the major player, but among themselves they behave as if they were independent, and this is really at the root of the mean field game theory. So this is a mean field interaction, and in for large system we expect propagation of chaos, namely that each single little guy becomes independent of everybody else. All right. All right, so now I want to finish with discussing another example, and this is an example borrowed from a small paper of Kolokolstov and Alemba and Souza. They have a small example of what they call a botnet cybersecurity model, and this could be a model of infection, of disease, or this is a simple model. The idea is that now we do not have stochastic differential equation, but we have a state space instead of being RD, it's a state space with four states, D, I, D, S, U, I, and U, S, for, so the situation is the following. So we all have a computer here, and you know, we may be worried they're going to have an election, oh no, you did that already, damn it. So we're going to have an election, and we may be afraid that some foreign power may try to mingle with our election, you know, that's something that, no, it never happens, I don't know. It's a hoax. It's a hoax. So we may worry about, you know, someone attacking our computer system. So the interest that I had in this particular model and the reason why I'm still working on this very simple model is the following, and I may not have much time to discuss it today, and I was trying to make a numerical computation for that. The question that I want to address is the following, and I will not be able to address it in full now. Just imagine that we use a mean field game model like Kolokolstov and Ben-Sussan. What this means is that we're going to let every single one of you selfishly, selfishly, makes a decision. I'm going to buy a computer protection, an anti-virus, and I'm going to install it. Who cares, you know, I never got infected before, I'm not going to be infected now. So everybody makes his own decision selfishly. You have some worry about what you have on your hard disk, you know, your attachment to your file, to your PDF, to, you know, pictures of your girlfriend or your mistress or whatever that is. You want to protect that, and you know, depending on the value you put on it, you know, you're going to make a selfish decision. Of course, you know, when you do in a mean field game, we're going to anticipate or we're going to try to estimate what the average of the people do. So we're going to look around and see, you know, if 90% of the people are infected in the room, you know, maybe I do not enter the room or I wear a mask, you know, I'm going to be very careful because the probability that it's going to be infected is higher. On the other hand, if no one is infected in this room, why would I buy, why would I attack antibiotics? There's no reason for me to do it because no one is going to infect me. No one is infected. Okay. So we're going to try to guess what is happening with these computers and we're going to make a decision. But the decision is very selfish. These controls in terms of the terminology are called distributed control. Namely, we make the decision on the basis of our state and nothing else. So this is what we're going to do. In cybersecurity, many, many, many, many papers have been written using game theory. But the way things are done is that we assume that we have a network of computers and there is a network manager, you know, your department hires some dude to take care of your computers and the manager decides, okay, so actually it doesn't ask you. It is installing, you know, an antivirus on your machines without you being aware of it. So the game is between an attacker, a hacker and a computer manager, but not the hacker and the network. Okay. So if you want to have a mean field game problem, what we're going to do is we're going to have a hacker, you know, attacking the computers and we're going to have a bunch of people making their own decision selfishly. All right. Selfishly. I insist selfishly. So that's what you do in mean field game. Now the issue is the following. And that was my question, you know, is it worth to spend the money and hire a computer manager for this network or should we leave it to people to do whatever they want? Okay. So there is a price to pay in a mean field game as opposed to a centralized decision by the computer manager or, you know, the minimal cost will not be as low if we let people deciding on their own selfishly as opposed to the cost to the society if we have a regulator making the decision for everybody. Okay. And the question is how different are they? You know, if the cost is only twice what it would have been, maybe I don't care. I'm not going to hire a network manager. You know, statistically. On the other hand, if the difference between the two is a factor of 10 or 100, then maybe I'm worried and I'm not going to let people optimize selfishly. So in game theory, this is what people have called the price of anarchy. Okay. So when people optimize selfishly, we can think of it as anarchy. I mean, some people do that. Actually the name was tossed by two Greek people. But that's what I want to study for this problem. So let's see. Oops. What are we going to do? So when the state space is finite, when the state space is finite, well, the dynamics are given by a Q matrix. And the Q matrix is going to be a function of two states. It's going to be a matrix. The rows have to sum up to zero. But we're going to have two parameters. One will be a parameter for the measure, the statistical distribution of the states of the other players, and then a control to affect the rate at which the state will change from one state to another. To make my life easier, I'm sure this is what already in the model of Kolokossof and Ben-Susan, you know, we're going to take the control to be zero one. Zero. It's not going to change anything. I'm happy with what is going on. One, I started panicking. I see some of my neighbors being infected. So I'm going to change my level of protection. I'm going to go from one protected to protected or protected to unprotected. What I emphasize here is that, again, we're going to take feedback Markovian control. So they're going to be function of the state XT, the state being, as I said, one of these four states. And if I look now at the evolution of the distribution, this is given by Falka-Planck Kolmogorov equation, which is almost a linear equation, except for the fact that the measure enters in the coefficient. So it is a form of McKean-Vlasov equation, even though it is for a continuous time Markov chain. We can write the Hamiltonian. We can minimize the Hamiltonian when we are lucky. If the state of control is finite, so that's going to be easy. We have a minimized Hamiltonian, and we also have an HAB equation. That's easy. In the particular case of this model of Kolmogorov, Sof and Ben-Sousson, if the control is zero, I do nothing. This is the transition from one state to another, dI, dS, uI, and on the horizontal, the columns are called dI, dS, uI, and uS. And these coefficients have some meaning, beta, or rate at which you get infected if you're defended, et cetera, et cetera. But you see the measure mu appearing in the transition in the Q matrix. So it is not going to be a typical linear Fokker-Planck equation. That's going to be a nonlinear Fokker-Planck equation. In any case, you can solve your mean field game problem. And if I plot here the time evolution of the distribution in equilibrium, distribution will be four probabilities. I have a probability to be defended and infected, defended and non-infected, not defend myself and be infected, and not defend myself and not be infected. If you start with equal probability of these four states, things change and evolve over time. If you start with a delta function, all the, I cannot read from here, but all the computers are in one specific state, they evolve, the distribution evolve in this way. What you can notice is that these distribution becomes constant over time. And so that's basically screaming for an ergodic model, a steady state model. And indeed, this is what Kolo-Kolsoff and Ben-Sussin studied, an infinite horizon ergodic model where they took one coefficient to infinity to make their life easier. But you can solve that completely. We can talk about the master equation and solve it and recover the same result. I'm not going to worry about that. So what I want to emphasize is just two words about this price of anarchy bound. In other words, you want to compare. So now let's imagine we get rid of the hacker. The hacker is simply a parameter in our model. We're going to look only at the mean field of the computer in the room. And so, and there is, I don't know, a 60% intensity of attack by the hacker. And we want to decide what is the cost to the group if we defend ourselves by not following a policy imposed by someone else? Or if there is a central planner making the decision for everybody and telling you, this is what you're going to do. This is how you're going to behave. And so, as I said, the cost to everybody will be lower if the central planner comes in. And the idea of this price of anarchy bounds is to find how much worse are we by letting everybody make their own decision. So just two minutes. If we are end players, end computer in the room, we can imagine that the state of our computer evolves according to a continuous time mark of chain, and each player is going to minimize a cost of this type. And here, this is the cost to player i, hence the state xit. And this is the empirical distribution of all the other computer. And sorry, what would be the cost to the group? So we're going to take a cost per player. So we're going to take one over end, the sum of the cost to all the players. And if you play around a little bit, then you take the limit as n tends to infinity. If there are no common noise, the empirical measure are going to converge to a single measure, and the social cost is going to be of this form. The running cost, including the measure, the terminal cost, including the measure, but now it's going to be integrated out because of this one over end sum from one to n with the same measure muti. So this is the social cost if everybody used the control phi, and if mu is the distribution of the state of the player. And so what we're going to have to do is decide how do we study this social cost. The mean field game problem is one way to solve it where the control phi and the measure muti are tied by a very tight constraint, while the social cost optimization will be leading to a very similar, very similar optimization, but the control phi and the measure mu will not be tied by such a tight constraint, and the minimum will be lower. And again, as I said, I didn't have time to go any further, but you have now, when you study the social cost problem, you have now a control of the McKinvass of type, even though it is purely deterministic, but it's in the space of measures, and you have to solve an HGB equation in the space of measures when the state space is finite, thing are easier, but I'm done. Obviously the guy is tall, big, young, so I'm not going to fight. I'm done. Thank you. Thank you very much. One quick question before lunch. I want to ask for the open loop strategy. I don't well understand, in fact, because the SDE for the representative agent is given in a weak sense, the Brownian WT. I'm not sure they understand the question yet. Yes, it's expressed to express the limit behavior in law, no? When you write the strategy, alpha, it depends on the path of the Brownian motion, W. For me, W, the Brownian is just a tool to write the SDE for the representative agent in a weak sense, I mean. So I didn't understand my question. The remark I was making is that if I think of a game or control problem where I'm going to actually implement the optimal control. So in other words, when I see the system evolve at time t, I want to be able to implement my control. I want to decide where the B's are going to go, how to change the velocity. I have to have a function of something that is hopefully available to me. And the question is that for open loop, you take a function of the path of the Brownian motion. And this is not something that we observe directly. Yes, we don't observe it. So mathematically, this is what, a progressively measurable process. So it's cool. There are spaces of these processes. We put an integrability condition, expectation of 0 to t of the square is finite, we have a nice, little bit of space, and we can do mathematically a lot of things. However, if you want to implement it practically or do it numerically, unless you're going to solve that during the summer school, this is a tough challenge. So this is what I say. However, mathematically, and the first papers we wrote with Francois on the subject, we were using open loop control and open loop Nash equilibria. But unfortunately, I believe this is not very practical. It's useful mathematically, but not always practical. Thank you. Thank you again. We. Thank you. Thank you. Thank you.
 Over the years I've been giving a few talks and each time I thank the organizer.  But this time I really mean I want to thank the organizer.  A few years ago, maybe over 40, in 69, I passed what some of you may still know,  aggregation, and with all the people from 68.  And in July I had two options.  I mean three options actually.  Going to teach in a high school somewhere in the middle of the country.  I was not excited.  And then I had an offer from Grenoble and an offer from Lumini.  Lumini was just being created.  So it was just a no-man's land and there was this half of this huge building of nine floors.  And people were pushing me, you know, Grenoble, you know, this is the high point of PDEs,  Malgrange is there, you should go blah blah blah.  And at that time I decided that now I'm not the kind of guy who is going to leave Marseille.  I'm going to stay in Marseille for the rest of my life.  Well, this afternoon I'll be happy 40 years later to go back to the Karang.  All right, so I'm going to talk about a mean field game and I didn't know, you know,  what would be covered in some of the tutorial lectures.  So I'm going to talk about something a little bit off the main beaten path.  And I'm going to talk about games with major and minor players.  And then at the end I will talk about something that I wanted really bad to talk about,  but for which I didn't have the time to actually do computation and prepare illustration for today.  You know, at my age if you start learning a new programming language it takes more time  than what you think.  All right, so I'm going to talk about mean field game with major and minor player.  And someone told me that this was working, but maybe not.  Oh, defy any logic.  This is a left-handed gadget, I guess, not a right-handed guy.  All right, so this is the typical starting point if the models are models  with the stochastic differential equation.  The second part of the talk I'm going to talk about an application  where the model will not be given by stochastic differential equation,  but at least this will give us a little bit of a sense of what we're talking about.  I'm just playing with this gadget, but it doesn't seem to be.  Why do I go back?  Here we go.  All right, so we have a major player and its state will be denoted by, with the superscript  zero and we have a generic player and there will be in large number,  and this is why I call it generic, which state will be denoted by xt.  And both states are evolving according to a stochastic differential equation,  they're ETO processes, but what is important to notice is that the control of the major  player, which will be denoted by alpha superscript zero, of course occurs in the dynamic of the  state of the major player.  This guy controls its own destiny, but it also appears in the dynamics of the minor players.  And so this is in this sense that the major player influences the minor players.  Also the state x0t of the major player appears in the dynamics of the state of the minor  player, so if you think in terms of financial application, you look at the financial system,  for example in the US, you have thousands and thousands of small banks, and then you  have a small number, finite number of systemically important institution, think of Goldman Sachs,  and whatever Goldman Sachs does is going to affect whatever all the other banks do.  On the other hand, whatever the local bank in Irvine, California does, is not going to  worry too much, Goldman.  So the dynamics of the major player state is not affected by the individual state of  the player, still you have this mu t entering in both stochastic differential equation,  and this mu t represents the empirical distribution of the states of the minor players.  So the major player feels the field of the minor players, it does not feel them individually,  Goldman as I said doesn't care about what the savings and loan in Irvine do, but on  the other hand, Goldman may worry about the proportion of savings and loans doing this  or doing that.  So that's how the system is mean field, it is mean field among the major, sorry, among  the minor players, and this field of minor players is taken into account by the major  player, but on the other hand, the major player state doesn't depend upon the individual  minor players.  So this creates this symmetry and we're going to assume that we have a large number, an  infinite number, continuum if you want of minor players and just one minor player, a  major player, and this major player could be multi-dimensional, in other words, you  could have Goldman, Morgan, Stanley, JP Morgan, but only in finite numbers.  Then each of these players, major or minor, incurs a certain cost and the cost as usual  is given by an expectation of the effect of a running cost, an integral from zero to the  terminal time capital T and a terminal cost, G0G, and again we have the same structure,  namely the cost to the major player depends upon the field, upon the measure of UT of  whatever the minor players do, but on the other hand, for the minor players, the cost  running or terminal depends upon what exactly the state of the major player is, X0T, or  even what actions the major player is taking.  Wunderbar.  All right, so I'm just going to give a few slides to propose a way to formulate the problems.  By the way, most everything that I'm going to say except that the hint at the calculation  which I couldn't finish is already in a beautiful book that Francois will sell you at a discount  if you behave well during the summer school.  So we're going to assume that the controls are first in open loop form.  Open loop control are very convenient and probabilists love them.  Practically I just believe they're nonsense, they're not practical, and in particular if  you want to do a numeric calculation like Eva's been explaining to us this morning,  you need to work with closed loop.  However, there are many mathematical results that you can first prove for open loop formulation  of the problem, and as I said, as probabilists, they are very convenient.  So the notion of open loop can be formulated here by saying that the control alpha 0 will  be a function of time and the entire path of the white noise of the state of the major  player.  The control of the minor player similarly will be a function of time and the entire  path of the white noise of driving the equation of the major player and the white noise driving  the equation of the minor players.  Who sees these white noise in this path?  I do not know.  So practically if I wanted to construct this open loop control and take that as an action,  I would have to see the entire path and in practice, as I say, unfortunately, we rarely  see them.  Sometimes we can, but it's very rare.  And the function of these paths will be functions phi 0 and phi, which are deterministic functions  defined on these spaces, the time, the path space, and taking value in sets A0 and A where  the controls take value.  All right, so when you formulate a major and minor model, you first look at the problem  of the major player and you're going to see how does the major player going to respond  to the field of minor player.  So you first assume that all the minus player, which are behaving identically, choose an  open loop control, so a function phi, which is going to be function of, as I say, time  and the two white noise paths.  And when all these minor players use this control, then the major player is facing the  following problem, the equations for a state is the same as before, and then we have an  equation now for the state of the minor player where the function phi comes in.  These two equations are coupled because, of course, W0 appears in both of them.  And now mu t is going to be taken to be the conditional law of xt given the path of the  white noise of the major player.  And so the major player problem is going to be to search for an optimal control, phi  0 star, this will be the optimum, minimizing is cost, minimizing is cost, where now the  cost depends upon the function phi, which is chosen by all the minor players.  But the points that I want to make here is that no matter what, the problem, the optimal  control that the major player has to solve to find its best response to the decision,  the behavior of this minor player is an optimal control of the McKean-Vlasov type because  the distribution of the state here that is only the marginal of xt enters into the dynamic  of the equation.  So if you formulate the best response of the major player in this way, the problem you  have here is an optimal control of McKean-Vlasov dynamics.  Now we want to look at the best response of the minor player to the major player.  So we're going to assume that the major player chooses a strategy of control alpha 0 given  by feedback function phi 0, which is a function of the entire path.  And we're going to assume that the field of minor players choose a feedback function phi,  which is now a function of two white noses.  And once this is done, the dynamics of the two states, x0 and xt, solve this stochastic  differential equation, which is a stochastic differential equation, again, of the McKean-Vlasov  type because the law of xt conditioned by the white noses w0 enters in this equation.  But this is not a control problem.  You solve this SDE, which is an SDE of the McKean-Vlasov type.  But once you solve this SDE, you use the law of this solution and you inject that into  the optimization that the minor player, the typical minor player should solve.  And now you have an optimal control of the regular type.  This is not of the McKean-Vlasov type.  And once you can solve it as a function of phi0 and phi, we're going to denote by phi  bar star the optimal open loop control, which is given by a function of the white noses  path.  Let's assume that this can be solved.  Let's even assume that it is unique.  And now finding a Nash equilibrium for a system of major and minor player is finding a  fixed point to the best response map, namely finding a set of couple of feedback function  phi0 and phi satisfying this equality.  So I want to emphasize the fact that I want to find a Nash equilibrium for the whole  system, including major player and minor player.  This is going to be in contrast with a simple example I will give, I will show a slide in  a minute.  So this, if I can solve that, will give me an open loop Nash equilibrium for the system.  What is the closed loop version of this problem?  I didn't solve it.  Of course, I just formulated the problem.  But the formulation of major and minor models of mean field game has been all over the map.  These models were introduced by Huang and then Kane and Noorian.  And many people have written paper on it, but I think there is a major confusion in  how this model should be set.  Anyway, so now if you want to have a closed loop version of this problem, you do exactly  the same thing where now the control for the major player and for the minor players are  given by feedback function, but now they're functions of the trajectory of the simple  path of the states.  So that still may be unreasonable to imagine that I can keep track of the entire path of  the state before I make my decision at time t, but at least it is more reasonable from  a practical point of view.  And now if you want to have a more COVID version, a version for which we can do actual computation  and write PDEs, et cetera, et cetera, you will now take the feedback function phi 0  and phi to be only function of the state at time t.  Now why do I bother so much to write all this annoying definition is the fact that in general,  these form of the definition of an open loop problem or closed loop problems play a major  role in game theory when you work with finite many players.  If you work with finite many player, looking for open loop equilibrium will lead to some  solution which can be different than closed loop Nash equilibria.  Even if the open loop Nash equilibrium which you find is in closed loop form.  So it's not enough to have an open loop Nash equilibrium in closed loop form to be able  to say I got a closed loop Nash equilibrium.  That's the case.  However, and this is not really a theorem, but it can be proved in many cases and it's  some sort of a folk theorem, when the size of the population grow and the number of players  tends to infinity, and this is basically the situation we set ourselves up when we do mean  field game, these differences disappear in the wash.  And this is why very often when you study mean field game, you don't care so much about  closed loop open loop because whether you tackle for one or the other, very often the  solutions are the same.  Unfortunately when you deal with major and minor players, that's not going to be the  case because the limit n tends to infinity which is going to give a lot of averaging  happening.  You remember the idiosyncratic white noise attached to each of the players, they're going  to force some sort of law of large number and you're going to have things going to random  nest which is going to disappear.  And so while the random nest disappear, the differences between the closed loop and open  loop disappear as well.  That's not the case here because the W0 and the X0 are going to remain.  We have one major player or finite number of major player, we have a large number of  minor players, they have their idiosyncratic noise.  When the size of the minor population grows to infinity, you get your averaging but you  never average out the noise of the major player and you saw that the measure that I was taking  were conditional measure.  So this averaging is not there and it is likely and we're going to see example that  closed loop and open loop Nash equilibrium still differ, still in a mean field limit.  So the best response being found in these classes of control, again finding a Nash equilibrium  is finding a fixed point.  So this is an example of a paper which was started years ago and which is still not finished  but where it looks like there is a major, it looks like there are minor players but it  is not a Nash equilibrium of this type that we would be looking for.  When you do contract theory, you have a principle and for my friend Ronea, let's see the regulator  of the European community, not the US because we didn't sign the Paris Accord, it's not  for us, this sort of thing.  So we have a regulator and the regulator basically talk to all the electric company, RWE, DF,  etc., and tell them, okay, so this is what we're going to do.  This is the contract that I give you.  You're going to be rewarded for reducing your emission, you're going to be penalized if you  screw up and you go over your cap, whatever the regulator decides.  So this is the major player and then you have the mass of minor player, all the little electric  operators, the little people like IDF, and so they're going to react to this regulation  and they're going to choose a behavior which depends upon the regulation set in place by  the regulator.  So you have also a symmetry between a small number of player, the big player being the  regulator and a large number of player, the electric operators, but now this is different.  The Nash equilibrium will be a Nash equilibrium among all the electric operators, not involving  the regulator.  The Nash equilibrium will be among the minor players only.  So that's a slightly different kind of problem than the one that I had before where before  I won a Nash equilibrium among everybody.  In other words, I won the major player not having any incentive to change his contract  once the minor players have chosen what to do.  All right, so when can we solve this mean field game?  We do not have many large classes of problems.  So I'm going to stick to two simple models for which we can actually analytically solve  the problems and do some numerics.  I'm going to first look at linear quadratic model and then I'm going to look at models  for which the state space is finite, in which case we will not have a stochastic differential  equation, but intuitively we can make sense of how things are working.  So a linear quadratic model will come in this form.  The, no, it's still not working.  The x0, the state of the major player, will satisfy a linear equation and I assume that  the volatility is constant.  And what is entering in this linear equation is x bar t, which is the conditional expectation  because it's linear the way the measure comes in will be only through its mean.  And so the minor players influence the major player decision only through the mean.  And their dynamics, the dx t, involves also their means, but they involve the dynamics  of the minor player state involve x0 t, which is the state of the major player.  And same thing for the cost except that now we take them quadratic, linear quadratic,  linear dynamic quadratic cost.  Okay, so you have the same type of equation.  Again the mean x t bar of the states of the minor players enters the cost of the major  player, but that's the only way the minor players will influence the dynamics and the  cost of the major player.  So the reason we can do that is that again, you know, we can write down the optimization  problem that I mentioned before.  You have linear dynamic quadratic cost, so this optimization problem reduced to big  matrix Riccati equation.  And so whether you do that through an FBSD or through another approach, you know, you  end up with big matrix Riccati equation and hopefully you can solve them.  And in this particular case we can actually solve them in the case of open loop directly  and we find a solution.  In the case of closed loop, things are a little bit more difficult.  However, if we search for controls in a specific form because we know in advance that the controls  are going to be linear in the states, so we look already for controls which are linear  in the states and the mean.  Now when we try to find that these controls satisfy the Nash equilibrium, we perturb that  with general control.  So it's really a Nash equilibrium that I will find, but I search for a Nash equilibrium  in this form, but that's not a restricted notion of a Nash equilibrium.  That's going to be a general Nash equilibrium.  And then again, you know, we can formulate the optimization problem in the form of a large  FBSD and of course they are fine and we solve them through solving a large matrix Riccati  equation.  However, in this particular case they're different.  So this means that the open loop and the closed loop solutions are different.  So that happens even for mean field game problem and that happened even for linear quadratic  case.  So as I said, one of the reasons why I went through these little song and dance was to  emphasize the fact that that's an issue, which are the important one.  I mentioned that already three or four times.  So I'm going to give you just a very simple naive application.  So let's imagine that you're following a beehive.  You want to see how the bees move from one location to another.  So how do they do that?  So the queen is getting old.  There is a new queen in the bunch.  So they want to get rid of the old queens.  So the queen leaves the group with a certain number of worker bees following the queen.  They didn't notice that there was a young and more beautiful queen.  So they follow the old queen.  And so they go and sit on a branch on a tree and they wait.  And then a few bees called the stricter bees, they go and they look around in the neighborhood.  They try to find a new location for the hive.  So once they think they found a new location, they come back to the tree where the old queen  and the workers are waiting there.  And they start dancing and dancing, trying to convince this group to follow them.  And eventually, one of them is dancing so well that the group decides, OK, we're going  to follow this stricter bee.  And they follow the stricter bee to the new location.  That's a mean field game problem, just kidding.  I'm going to pretend that's a mean field game problem and that's a linear quadratic  mean field game problem.  That's a linear quadratic mean field game problem with a major and minor player.  And we're going to see how it works.  So we're going to model only the velocity because it's simpler if you model the velocity  and the position.  It's a little bit more complicated mathematically because you have hyperliptic diffusion instead  of diffusion and things are not as simple.  So we're going to denote by v0n the velocity of the stricter bee, the bee which is going  to go to the final destination, which knows where it is and which wants to drag the old  queen and the bunch to this new location.  So that's going to be the velocity of the stricter bee.  And the control is alpha 0 up to noise.  The stricter bee controls its own velocity.  We have n, we have n capital N bees with the queen.  And so they control the velocity, alpha it, and they have their own idiosyncratic noise.  What is the cost to the stricter bee?  So you have three components to the cost.  Then we take a linear combination of these three components.  The first component is the square of the difference between the velocity of the stricter bee and  the optimal velocity it would take to go from point A to point B.  So in other words, this new t here is a deterministic function which if the bee used that as a velocity  it would take from point A to point B. Then the second term is I want the velocity of  the stricter bee to be not very different from the average velocity of the group because  the stricter bee wants to pull the group to the new location.  And finally, just a kinetic energy term, you want not to be too tired and exhausted.  You want to end up where you want to go.  So that's the cost of the stricter bee.  Now what are the cost of the little worker bees?  So they have again a cost with three components.  The first component is that they want their velocity to be similar to the velocity of  the stricter bee.  So they want to follow the stricter bee.  They decided to do that so they want to do it.  So that's the first term.  The second term is that they want to stay in a group.  So they want their individual velocity not to be much different than the average velocity  of the group.  And finally again, they want to end up where they want to be.  Namely, they do not want to get exhausted too quickly.  So that's the finite player game.  You write the limit as n10 to infinity as a mean field game.  And this is one example of numerics that you would get.  So in this particular case, you know, this is silly.  There is no new location.  I take the velocity to be circular.  Okay, so this is the velocity in use of t.  And so you see in black the trajectory of a stricter bee following this velocity.  And then at a distance, remember, the position is not part of the model.  So if they start from different places, there is a plot of 10 trajectories of 10 of these  worker bees.  And you know, they try to move around in circle, move around in circle.  Again the position doesn't matter.  But this is for a certain number, a certain set of coefficient, the relative value of  the penalty.  But if you start monkey around with these coefficients, you know, even though the velocity, the optimal  velocity is the same, well, you know, the stricter bee is not even going to follow that.  And the worker bees are not even going to follow it.  Okay, so that's a typical example.  And there was another slide where, you know, the velocity in you was linear, and then you  could see also that the worker bees were following.  So you're going to have to buy the book to find the other plot.  And let me tell you how outrageous it is.  I mean, France is not even as upset as I am.  But you know, they want to sell this crap for $149 a volume.  Who is going to buy it?  You know, if you have, I mean, I know people have ways to do it.  Just get a copy, you don't have to buy it because, you know, this is ridiculous.  $149, so multiply by two, that's basically $300.  It's just ridiculous.  But you would have more pictures.  All right.  So that's what it is.  So this plot is an illustration of what's really behind mean field game.  Stefan, when do I have to close?  You have to close it.  Okay.  Okay.  All right.  So this is an illustration of what's behind the theory of mean field game, which is based  on a result you're going to hear about over and over and over during the week and presumably  during the many weeks if you stick around is the propagation of chaos.  Here it's a simple form of conditional propagation of chaos in the following sense.  What I did here is I took five workabees, five minor players, okay, and in condition  I generate one path for the white noise of the major player, namely the street cubby,  okay.  So this, what we're seeing here is conditioned, this is conditioned on the white noise of  the major player, okay.  But condition on that, I take my five street cubbies and I look only at a group of five  street cubbies.  I simulate their position and I compute the correlation matrix.  So if you compute the correlation matrix, you see that you have, so it's a five by five  matrix, and on the diagonal you have high terms, they should be one, and outside the  diagonal it is still relatively high, but it's definitely non-zero.  Now if you, if I redo the same thing, but my five minor players, I looked at them in  a group of ten minor players, okay.  So I redo the same simulation.  I have now ten minor players, only one path of fixed path of the white noise of the major  player, and I do the same thing and I compute the correlation, and it is much lighter outside  the diagonal.  And now if I take 20, and if I take 50, and if I take 100, you see that my correlation  matrix is diagonal.  Given the fact that we are linear quadratic, this is a Gaussian model, so this means that  they're independent.  So in other words, you know, when the bath, when the field of particle is larger and larger,  a fixed number of particles behave exactly as if they were independent.  Here it's conditioned on the white noise of the major player, but among themselves they  behave as if they were independent, and this is really at the root of the mean field game  theory.  So this is a mean field interaction, and in for large system we expect propagation of  chaos, namely that each single little guy becomes independent of everybody else.  All right.  All right, so now I want to finish with discussing another example, and this is an example borrowed  from a small paper of Kolokolstov and Alemba and Souza.  They have a small example of what they call a botnet cybersecurity model, and this could  be a model of infection, of disease, or this is a simple model.  The idea is that now we do not have stochastic differential equation, but we have a state  space instead of being RD, it's a state space with four states, D, I, D, S, U, I, and U,  S, for, so the situation is the following.  So we all have a computer here, and you know, we may be worried they're going to have an  election, oh no, you did that already, damn it.  So we're going to have an election, and we may be afraid that some foreign power may try  to mingle with our election, you know, that's something that, no, it never happens, I don't  know.  It's a hoax.  It's a hoax.  So we may worry about, you know, someone attacking our computer system.  So the interest that I had in this particular model and the reason why I'm still working  on this very simple model is the following, and I may not have much time to discuss it  today, and I was trying to make a numerical computation for that.  The question that I want to address is the following, and I will not be able to address  it in full now.  Just imagine that we use a mean field game model like Kolokolstov and Ben-Sussan.  What this means is that we're going to let every single one of you selfishly, selfishly,  makes a decision.  I'm going to buy a computer protection, an anti-virus, and I'm going to install it.  Who cares, you know, I never got infected before, I'm not going to be infected now.  So everybody makes his own decision selfishly.  You have some worry about what you have on your hard disk, you know, your attachment  to your file, to your PDF, to, you know, pictures of your girlfriend or your mistress  or whatever that is.  You want to protect that, and you know, depending on the value you put on it, you know, you're  going to make a selfish decision.  Of course, you know, when you do in a mean field game, we're going to anticipate or we're  going to try to estimate what the average of the people do.  So we're going to look around and see, you know, if 90% of the people are infected in  the room, you know, maybe I do not enter the room or I wear a mask, you know, I'm going  to be very careful because the probability that it's going to be infected is higher.  On the other hand, if no one is infected in this room, why would I buy, why would I attack  antibiotics?  There's no reason for me to do it because no one is going to infect me.  No one is infected.  Okay.  So we're going to try to guess what is happening with these computers and we're going to make  a decision.  But the decision is very selfish.  These controls in terms of the terminology are called distributed control.  Namely, we make the decision on the basis of our state and nothing else.  So this is what we're going to do.  In cybersecurity, many, many, many, many papers have been written using game theory.  But the way things are done is that we assume that we have a network of computers and there  is a network manager, you know, your department hires some dude to take care of your computers  and the manager decides, okay, so actually it doesn't ask you.  It is installing, you know, an antivirus on your machines without you being aware of  it.  So the game is between an attacker, a hacker and a computer manager, but not the hacker  and the network.  Okay.  So if you want to have a mean field game problem, what we're going to do is we're going to  have a hacker, you know, attacking the computers and we're going to have a bunch of people  making their own decision selfishly.  All right.  Selfishly.  I insist selfishly.  So that's what you do in mean field game.  Now the issue is the following.  And that was my question, you know, is it worth to spend the money and hire a computer  manager for this network or should we leave it to people to do whatever they want?  Okay.  So there is a price to pay in a mean field game as opposed to a centralized decision  by the computer manager or, you know, the minimal cost will not be as low if we let  people deciding on their own selfishly as opposed to the cost to the society if we have  a regulator making the decision for everybody.  Okay.  And the question is how different are they?  You know, if the cost is only twice what it would have been, maybe I don't care.  I'm not going to hire a network manager.  You know, statistically.  On the other hand, if the difference between the two is a factor of 10 or 100, then maybe  I'm worried and I'm not going to let people optimize selfishly.  So in game theory, this is what people have called the price of anarchy.  Okay.  So when people optimize selfishly, we can think of it as anarchy.  I mean, some people do that.  Actually the name was tossed by two Greek people.  But that's what I want to study for this problem.  So let's see.  Oops.  What are we going to do?  So when the state space is finite, when the state space is finite, well, the dynamics  are given by a Q matrix.  And the Q matrix is going to be a function of two states.  It's going to be a matrix.  The rows have to sum up to zero.  But we're going to have two parameters.  One will be a parameter for the measure, the statistical distribution of the states of  the other players, and then a control to affect the rate at which the state will change from  one state to another.  To make my life easier, I'm sure this is what already in the model of Kolokossof and  Ben-Susan, you know, we're going to take the control to be zero one.  Zero.  It's not going to change anything.  I'm happy with what is going on.  One, I started panicking.  I see some of my neighbors being infected.  So I'm going to change my level of protection.  I'm going to go from one protected to protected or protected to unprotected.  What I emphasize here is that, again, we're going to take feedback Markovian control.  So they're going to be function of the state XT, the state being, as I said, one of these  four states.  And if I look now at the evolution of the distribution, this is given by Falka-Planck  Kolmogorov equation, which is almost a linear equation, except for the fact that the measure  enters in the coefficient.  So it is a form of McKean-Vlasov equation, even though it is for a continuous time Markov  chain.  We can write the Hamiltonian.  We can minimize the Hamiltonian when we are lucky.  If the state of control is finite, so that's going to be easy.  We have a minimized Hamiltonian, and we also have an HAB equation.  That's easy.  In the particular case of this model of Kolmogorov, Sof and Ben-Sousson, if the control is zero,  I do nothing.  This is the transition from one state to another, dI, dS, uI, and on the horizontal, the columns  are called dI, dS, uI, and uS.  And these coefficients have some meaning, beta, or rate at which you get infected if  you're defended, et cetera, et cetera.  But you see the measure mu appearing in the transition in the Q matrix.  So it is not going to be a typical linear Fokker-Planck equation.  That's going to be a nonlinear Fokker-Planck equation.  In any case, you can solve your mean field game problem.  And if I plot here the time evolution of the distribution in equilibrium, distribution  will be four probabilities.  I have a probability to be defended and infected, defended and non-infected, not defend myself  and be infected, and not defend myself and not be infected.  If you start with equal probability of these four states, things change and evolve over  time.  If you start with a delta function, all the, I cannot read from here, but all the computers  are in one specific state, they evolve, the distribution evolve in this way.  What you can notice is that these distribution becomes constant over time.  And so that's basically screaming for an ergodic model, a steady state model.  And indeed, this is what Kolo-Kolsoff and Ben-Sussin studied, an infinite horizon ergodic model  where they took one coefficient to infinity to make their life easier.  But you can solve that completely.  We can talk about the master equation and solve it and recover the same result.  I'm not going to worry about that.  So what I want to emphasize is just two words about this price of anarchy bound.  In other words, you want to compare.  So now let's imagine we get rid of the hacker.  The hacker is simply a parameter in our model.  We're going to look only at the mean field of the computer in the room.  And so, and there is, I don't know, a 60% intensity of attack by the hacker.  And we want to decide what is the cost to the group if we defend ourselves by not following  a policy imposed by someone else?  Or if there is a central planner making the decision for everybody and telling you, this  is what you're going to do.  This is how you're going to behave.  And so, as I said, the cost to everybody will be lower if the central planner comes in.  And the idea of this price of anarchy bounds is to find how much worse are we by letting  everybody make their own decision.  So just two minutes.  If we are end players, end computer in the room, we can imagine that the state of our  computer evolves according to a continuous time mark of chain, and each player is going  to minimize a cost of this type.  And here, this is the cost to player i, hence the state xit.  And this is the empirical distribution of all the other computer.  And sorry, what would be the cost to the group?  So we're going to take a cost per player.  So we're going to take one over end, the sum of the cost to all the players.  And if you play around a little bit, then you take the limit as n tends to infinity.  If there are no common noise, the empirical measure are going to converge to a single  measure, and the social cost is going to be of this form.  The running cost, including the measure, the terminal cost, including the measure, but  now it's going to be integrated out because of this one over end sum from one to n with  the same measure muti.  So this is the social cost if everybody used the control phi, and if mu is the distribution  of the state of the player.  And so what we're going to have to do is decide how do we study this social cost.  The mean field game problem is one way to solve it where the control phi and the measure  muti are tied by a very tight constraint, while the social cost optimization will be  leading to a very similar, very similar optimization, but the control phi and the measure mu will  not be tied by such a tight constraint, and the minimum will be lower.  And again, as I said, I didn't have time to go any further, but you have now, when you  study the social cost problem, you have now a control of the McKinvass of type, even though  it is purely deterministic, but it's in the space of measures, and you have to solve an  HGB equation in the space of measures when the state space is finite, thing are easier,  but I'm done.  Obviously the guy is tall, big, young, so I'm not going to fight.  I'm done.  Thank you.  Thank you very much.  One quick question before lunch.  I want to ask for the open loop strategy.  I don't well understand, in fact, because the SDE for the representative agent is given  in a weak sense, the Brownian WT.  I'm not sure they understand the question yet.  Yes, it's expressed to express the limit behavior in law, no?  When you write the strategy, alpha, it depends on the path of the Brownian motion, W. For  me, W, the Brownian is just a tool to write the SDE for the representative agent in a  weak sense, I mean.  So I didn't understand my question.  The remark I was making is that if I think of a game or control problem where I'm going  to actually implement the optimal control.  So in other words, when I see the system evolve at time t, I want to be able to implement  my control.  I want to decide where the B's are going to go, how to change the velocity.  I have to have a function of something that is hopefully available to me.  And the question is that for open loop, you take a function of the path of the Brownian  motion.  And this is not something that we observe directly.  Yes, we don't observe it.  So mathematically, this is what, a progressively measurable process.  So it's cool.  There are spaces of these processes.  We put an integrability condition, expectation of 0 to t of the square is finite, we have  a nice, little bit of space, and we can do mathematically a lot of things.  However, if you want to implement it practically or do it numerically, unless you're going  to solve that during the summer school, this is a tough challenge.  So this is what I say.  However, mathematically, and the first papers we wrote with Francois on the subject, we  were using open loop control and open loop Nash equilibria.  But unfortunately, I believe this is not very practical.  It's useful mathematically, but not always practical.  Thank you.  Thank you again.  We.  Thank you.  Thank you.  Thank you.
2017-07-17_Carmona_0_47.png 2017-07-17_Carmona_48_95.png 2017-07-17_Carmona_96_155.png 2017-07-17_Carmona_156_203.png 2017-07-17_Carmona_204_251.png 2017-07-17_Carmona_252_395.png 2017-07-17_Carmona_396_1739.png 2017-07-17_Carmona_1740_2327.png 2017-07-17_Carmona_2328_3887.png 2017-07-17_Carmona_3888_4583.png 2017-07-17_Carmona_4584_4667.png 2017-07-17_Carmona_4668_4739.png 2017-07-17_Carmona_4740_5003.png 2017-07-17_Carmona_5004_5051.png 2017-07-17_Carmona_5052_5111.png 2017-07-17_Carmona_5112_5159.png 2017-07-17_Carmona_5160_6671.png 2017-07-17_Carmona_6672_7775.png 2017-07-17_Carmona_7776_7943.png 2017-07-17_Carmona_7944_8447.png 2017-07-17_Carmona_8448_8615.png 2017-07-17_Carmona_8616_9839.png 2017-07-17_Carmona_9840_11075.png 2017-07-17_Carmona_11076_12071.png 2017-07-17_Carmona_12072_14255.png 2017-07-17_Carmona_14256_14303.png 2017-07-17_Carmona_14304_14975.png 2017-07-17_Carmona_14976_17627.png 2017-07-17_Carmona_17628_17831.png 2017-07-17_Carmona_17832_18995.png 2017-07-17_Carmona_18996_20699.png 2017-07-17_Carmona_20700_21983.png 2017-07-17_Carmona_21984_23075.png 2017-07-17_Carmona_23076_29855.png 2017-07-17_Carmona_29856_30179.png 2017-07-17_Carmona_30180_33695.png 2017-07-17_Carmona_33696_36683.png 2017-07-17_Carmona_36684_39695.png 2017-07-17_Carmona_39696_40223.png 2017-07-17_Carmona_40224_40367.png 2017-07-17_Carmona_40368_44423.png 2017-07-17_Carmona_44424_45695.png 2017-07-17_Carmona_45696_45923.png 2017-07-17_Carmona_45924_46967.png 2017-07-17_Carmona_46968_47999.png 2017-07-17_Carmona_48000_48599.png 2017-07-17_Carmona_48600_49031.png 2017-07-17_Carmona_49032_49175.png 2017-07-17_Carmona_49176_50435.png 2017-07-17_Carmona_50436_55883.png 2017-07-17_Carmona_55884_56003.png 2017-07-17_Carmona_56004_56639.png 2017-07-17_Carmona_56640_57167.png 2017-07-17_Carmona_57168_59087.png 2017-07-17_Carmona_59088_59231.png 2017-07-17_Carmona_59232_61883.png 2017-07-17_Carmona_61884_62315.png 2017-07-17_Carmona_62316_63647.png 2017-07-17_Carmona_63648_65423.png 2017-07-17_Carmona_65424_66719.png 2017-07-17_Carmona_66720_68183.png 2017-07-17_Carmona_68184_68543.png 2017-07-17_Carmona_68544_69695.png 2017-07-17_Carmona_69696_69755.png 2017-07-17_Carmona_69756_71687.png 2017-07-17_Carmona_71688_71735.png 2017-07-17_Carmona_71736_71843.png 2017-07-17_Carmona_71844_71891.png 2017-07-17_Carmona_71892_73583.png 2017-07-17_Carmona_73584_74483.png 2017-07-17_Carmona_74484_74531.png 2017-07-17_Carmona_74532_75779.png 2017-07-17_Carmona_75780_75851.png 2017-07-17_Carmona_75852_76391.png 2017-07-17_Carmona_76392_76931.png 2017-07-17_Carmona_76932_79439.png 2017-07-17_Carmona_79440_81911.png 2017-07-17_Carmona_81912_82139.png 2017-07-17_Carmona_82140_82211.png 2017-07-17_Carmona_82212_82259.png 2017-07-17_Carmona_82260_82307.png 2017-07-17_Carmona_82308_82384.png