Mgmt Config: The Road to 0.1
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Shubin, James
Mgmt is a real-time automation tool that is fast and safe. As we get closer to a 0.1 release that we'll recommend as "production ready", we'll look at the last remaining features that we're aiming to land by then. We'll explain and demo the import and module system, classes, and native functions. We'll also show some exciting real-time demos that include scheduling, finite state machines, and remote execution. Mgmt is a real-time automation tool that is fast and safe. As we get closer to a 0.1 release that we'll recommend as "production ready", we'll look at the last remaining features that we're aiming to land by then. We'll explain and demo the import and module system, classes, and native functions. We'll also show some exciting real-time demos that include scheduling, finite state machines, and remote execution. A number of blog posts on the subject are available: Attendees are encouraged to read some before the talk if they want a preview!
 So I'm a hacker, I work on config management stuff. I write a technical blog called the Technical Blog of James. Who's seen it? Just raise your hand. Oh wow, if you haven't seen it, just raise your hand anyway so I seem really popular. Excellent, thank you. I'm actually a physiologist by training and I do all sorts of DevOps stuff. So if you want to talk cardiology, let me know. I used to work on a lot of puppet stuff a long time ago. Nowadays it seems that every day we're kind of yamling, like Ansible and Kubernetes and all this stuff. There's this yaml camp, Twitter account. And is this what we want? Is this how you want to build your infra with yaml? Have you heard of, there's a yaml assembler? What is going on in the world? Basically you just, someone writes some code and you blast them yaml at it and hopefully you get some sort of infrastructure. I don't know. Who uses these things? There's some usefulness but I'm a little skeptical. And so the real question is, do we want to be yaml programmers? All right, let's see. This is my answer to if we want to be yaml programmers. It's the nope, it's definite nope. I'm sorry. YAML is useful for some things but you need the right tool for the right job. And for building real time distributed systems and infrastructures, I think we really need a language to be more expressive and also safe. So I have this software called MGMT that I've been working on. It's got two parts. It's got an engine and a language which I'm going to talk about today. Basically in the engine, you can see the stuff online I've talked about before. It's kind of like puppet in that there's, sorry, it's kind of like puppet in that you have a graph of resources that you run through like a dag. But we can actually run that graph in parallel which is really useful. It's event driven which I'll show you also. And it works as a distributed systems. That's kind of cool. How's the sound alter? So do you want to see a quick demo of that? I'm going to sit down a lot so don't worry if you're scared. I'm still here. I'll show you just the classic demo. Basically I just run MGMT on the left here. And I just delete all this stuff. So basically I've asked MGMT to make one file and you can see it's just kind of like puppet. Creates a file declaratively. But the cool thing about this is if you remove the file, MGMT is actually running in real time, it detects this and it puts the file right back. So you do this, remove the file, comes right back. And it works so quickly you can really even just do this and cap the file. And as fast as you go, it's always coming back. Thank you. But wait, hold your applause. So you can actually run this watch command to just ask it to do this as fast as possible and you can see it's still just going really, really quickly. So I'm glad you like that. So pause is awesome, but we do this for every resource. So files, packages, services, virtual machine, containers, AWS resources and so on. So it's just a simple example, extrapolates whatever you want to do. So I want to talk more about the language. I'm sorry I'm going really fast. I have a lot of stuff to cover. And so we actually built a language to describe these systems over time. Some of the things we want in a language, basically something, the reason why we don't want YAML, we want something that's safe. We want it to be very powerful. So our language is reactive, which lets us do some cool stuff which you'll see. And we want something that's easy to reason about. So you want hopefully to write a few lines of very safe code and have that describe a very complicated infrastructure. And the reason we want this is because if you make an off by one error or like a YAML type or whatever, you don't want to blow away a whole data center, right? Right? Yes, maybe you do, but if you do, we're going to do it efficiently. So anyways. So you want to see some demos? Come on, you want to do some demos? All right, thank you. So here's the first demo I'm going to show you. I'm just going to start running this over here. So I'm going to run MGMT on the left. And basically it's a bit abstract, so I have to apologize for that, but I'm just writing some code that's going to run. And the output is just going to build some strings and store it in a text file. And the reason I'm just storing it in a text file is because that's easy to see what's happening. And that text file I'm just going to pull like I did with the watch so you can see what's happening. So here's the code. Oops. Oops. Sorry. One second. So here's the code. And I'm just going to run it on the left. And if you see, I just run this little watch, I've asked it to print out a few things. So I've asked it to print out the load on the current system. So load is a function that comes with the language. We also have date time, which is a built-in function. And this is a reactive programming language. So these functions generate a stream of values. So every second, date time has a new value it wants to produce. And the language intelligently reevaluates everything that needs to be reevaluated whenever something changes. So basically take these values, we put them in this variable, put them all into a big struct and then print out that struct in this file. And if you see, we actually can see, is it big enough? Can you see in the back? Yeah? Good. You can actually see what's happening. You can see the seconds are incrementing every second. And you can actually see the load on my machine in real time. And in fact, just to show you that any sort of data can be used as input, I have this VU function here. And this is actually using my microphone in my laptop in real time and recording the room audio and transforming that into a little graphical representation. So if I make a lot of noise, you can see it goes up. So if we're really quiet and then you make a lot of noise, let's see if it works. Very good. So I've made this joke before and I've talked about this particular example before. But imagine you have a server room and you put some microphones around. And if there's a lot of loud noise, because people are fighting, you could set things read only for an hour and wait till the fight stops and people settle down. I mean, these are the sort of things we need to do. We have to stop responding to pages and monitoring problems. We need to program the logic in advance and let some sort of engine and language deal with it. Does that make sense? Okay, I want like 300 contributors at the end of this talk. So that's this demo. Do you want to see some more demos or have you had enough? All right, cool. So it's a big thing to write this big project. So we just actually landed import and modules. So you can actually write whole modules of code now, which sounds silly. It's not great for demos, but just to show you basically the way it works is you have an import. So you import a module or another directory or even like an FQDN. And then all of those things actually give you a separate namespace for each one. So if you were to import FMT, you could actually run FMT.printf. So who's ever used Puppet before? Puppet users. Puppet had this horrible problem where if you had two modules named SSH, you couldn't have two. You'd have to rename one in the code and it was a disaster. So now you could actually have like, hopefully not, but 10 modules named SSH. And if you really needed to use both at the same time, you could like import module as name one and other module as name two. That sort of thing. Makes sense? Silly stuff. This is sort of like future of MGMT. So all right, so that's that. You want to see another demo? Yeah? Okay. You sure? So I have this really cool demo. So what I'm going to do, this is a bit complicated. So first I'm going to be using a runMGMT on my laptop over here. And what it's going to do, it's going to actually start up a new VM. Okay? Let's see if it worked. And so that's great. So now I'm going to actually log in to that VM to see what's happening. So that's the VM on the right side, which is starting up. Great. I'm just going to log in so you can see what's happening. Now you could have it automatically start MGMT on first boot, but I'm not going to do that because I want you to actually see the log output. And so I made a little script here just to run MGMT. So now inside the VM, I'm also running MGMT. Right? Normally you would do this automatically. So MGMT is running. It's running some different code. And here's what's actually going to happen. Oops. I actually have to kill this for a second. I'm going to run this inside of screen for a second. So we can just making a screen session so we can run this twice. So I'm running MGMT. And over here in the VM, I'm going to actually run this little script. It's really small. Display CPUs, which is just going to run a little loop to print out how many virtual CPUs are on this VM. Okay? So on the right, you can see I have MGMT code which prints out how many CPUs it sees. And I also have LS CPU which is showing you that there's one virtual CPU. Makes sense so far? Okay. So here's the fun part. Now it gets exciting. So the MGMT code on the left, on my laptop, it is actually, there's a reactive variable which is reading a text file. And so if I go over here, cat CPU count, it just has a number one in it. So if I actually echo two to this file, MGMT is going to notice. It's going to suck in that file into the MGMT code. It's going to see, oh, the number of CPU, the number is two now. And then it's going to take that number and set it as the VRT resource CPU count. And what that's going to do, it's going to hot plug a CPU hopefully. Now when it does that, inside MGMT, inside the VM, MGMT is actually going to notice that that CPU count has changed. And it's going to also use that variable and write out to the file that it has hot plugged in new CPU. Does that make sense? Let's hope it works. You want to see? Ready? Did it work? I don't know. Actually, it didn't work. Oh, sorry. I cat it. Sorry. Wrong thing. Echo two. There we go. There we go. Cool. So it changed. So we can put it to four. Thank you. So watch. Again, over here on the left, MGMT is just waiting and sleeping until that file gets to change. You set it to four. It quickly gets to four. MGMT inside says it's asking the kernel, hey, how many CPUs are there? It sees that there's new CPUs and then it runs some code differently. And just to show you how fast this actually works, I have this little script. Yeah? Yeah, thanks. That's Walter Heck, everyone. Pull in data. Check them out. So I made a little script. Just a crappy little shell script. Instead of echoing to that file, just like echoing every time I press a button. So now we can very quickly press up and down like this and it will change in real time. So that's the sort of speed. Now think about this. Why is this useful? Like, I'm just doing this as a silly example. But in real time, imagine you're some VM host. You have a whole bunch of big servers. A bunch of them are sleeping. And then you put all your customers, VMs, all on the same host until the load goes up. And then you sort of start adding more CPUs to deal with that load, shrinking other CPUs. Inside the VMs, you can do that as well. So you can really start programming real time by the second decisions across your whole infrastructure. Good idea? Yeah? Yeah? Some people are like, I'm not sure. You'll get there. So, yeah, any emergency questions about this? I got more demos if you want to see. More demos? All right, cool. So I'm just going to kill this. So if I actually shut it down without killing MGMT, it will turn it back on. So you got to shut this down first. So I showed you this magic stuff. So you can hot plug and hot unplug CPUs very, very quickly. This is all using libvert. And the CPU count is actually a function in the language that actually lets you decide. Let's do some scheduling. How are you, people say it. So I said that MGMT lets you have a distributed system of different machines. So let's say you want to do some task on N out of M of the machines. So what we actually do is we have a language function for that. It looks like this. Very simple. We have this schedule function. You give it some options as input. So for example, what kind of schedule you want to use? Here, this is just a round robin scheduler. How many machines do you want to use? How many machines do you want to pick out of the pool? So here's two. And I have a TTL of 10. And then what this function returns, it actually does behind the scenes all this raft distributed system magic. So you don't have to worry about it. And it returns a variable that contains which hosts were chosen. And each host will look through that variable and say, hey, am I one of the hosts in this list? If so, I'm going to do some stuff. If not, I wasn't chosen. Make sense? So I'm just going to run this. I'm just going to run a whole bunch of MGMTs on the left here. I'm just going to show you. So I'm just going to run this watch. We can actually do tail dash. F might be prettier. Oh, wait. The tail or what? Yeah. So I ran the first one. And this is basically the output of that first machine. Again, I'm just asking it, what did you choose as the scheduling decision? So you want to add a second one. Just start it up. And now there's two machines available. So H1 has decided if you're in H1 or H2, then you should do stuff. H2 has also decided you're going to be scheduled. Does that make sense so far? So every machine that joins the cluster is going to say, I'm part of the cluster. Do you want to use me? And the scheduler is going to decide, yes, we're going to use H1 and H2. And that should be consistent. If we start a third machine up like this, watch what happens. This machine starts up. And you can see it also is part of the cluster. And it knows that everyone has decided we're picking up two hosts, H1 and H2. All right. With me so far. So watch what happens if I shut down H2. You see instantly the first machine said, OK, well, now we've got to switch to H3 because we need to pick two hosts. And there was only two that are available now. And watch, machines two is actually dead. That file is just left over. When I start up H2 again, you can see that it comes back and it joins the existing scheduling decision. So it doesn't flip flop. And you can do the same thing. If you were to kill H3 now, you'll see it'll shut down. And these machines picked H2. Make sense? We'll start up H3 again just to show you one other quick thing. So if I were actually to forcefully unplug H2, I just forced killing it, you'll see that actually stays at H2 because there's that TTL timeout. So it's about 10 seconds. And in a few seconds, you should see it switched H3. Right. Perfect. You get that? Is that cool? Yeah. So what is this useful for? Anybody have an idea? So think about this. Let's say you wanted to use some crazy scheduler thing like Kubernetes or whatever. You don't need to use this huge custom thing. You can actually just describe what you want to have happen on which machines. It's just basically a little class which start VMs or start containers or do whatever and have a simple function like this build out your logic. You don't have to guess what parameters go in a YAML file. You don't need to know all sorts of fancy stuff. You describe your infrastructure as code and because it's running in real time in the special language, all that fancy stuff sort of happens automatically. That make sense? So you have to use your bit of imagination, but hopefully that's useful. You want to see more or you had enough? A little. It's like a quiet more. I can do up to two more demos maybe. All right. I'm just going to show you another language feature. Let me just actually kill all these really quickly. I'm just going to kill them violently. So I have this quick example. I showed you in the language about how every value is a sequence over time, but because we have this stream of values, we can actually look back in the past and see what previous values were. So in this case, we have date time and it just stored in a variable. And then we can actually take that variable and use curly brackets around it. So 0, 1, 2, 3, 4 to get the last or previous value and so on. And you can actually see those values over time. So I take all those values and print it in a file. We get the same sort of thing. I show you the contents. You get the same sort of thing here where you see the current time and then the time a second ago and the time a second ago over and forever. So if you use this information, you can actually do some clever things like was the value previously smaller? Are they going up? Are they going down? Is the load going up? Maybe I should be worried because the slope is really high. So I want to start up some machines to get ready for that. Well, quickly, jump in. You missed the fun parts, but there's more fun. Hop in. Foss stem compression. If there's room, just squeeze over. So these two kind of things are built into the language and their core primitives. So why are these useful? So the reason these are useful is because of this. I've shown this example before also. What does this look like, anyone? It's a thermostat. I took this photo with my parents' crappy camera in Canada. You can see it's in Celsius, the correct international units. And I, yeah, thank you. I have a clearer photo from the Internet in some sort of weird units. I don't know what they mean, but just to show you the idea. And these things, what? I can't hear you, but a sauna. Yeah, maybe that's funny. I make the jokes here. What's your name? No MGMT for you. Okay, so it's a thermostat. And thermostats have an interesting property. Does anyone know what it is? Or has anyone seen my talk before? I haven't seen it. What? Historesis. Someone pronounced it wrong, but you're all right. It's hysteresis. And all that really means is that if you had some sort of threshold, when you hit that threshold and you turned off the heat or got to that threshold and turned on the heat, when you're at that point, you don't want to flick the heat on and off repeatedly. We just go click, click, click, click, click, click, click, click, click, and we'll break the thing. So that's unfortunate. So hysteresis allows us to actually have a bit of a delay before we click back on after we go through a number. So I'm just going to show you a demo of this. I have a hysteresis example. So this example, again, it's about ten lines of MGMT code. What we do is we have this demo where it's going to start up some VMs and it's going to start up two VMs as long as the load on the system is below 1.5. And when it gets to 1.5, it's going to shut down one of the VMs. And just to show you all this happening, you can see the two VMs right here. I don't know why this guy is still here. It's just been on my system forever. And watch what happens when we increase the load above 1.5. You can see I'm actually printing out that value right here. So 1.5 is a threshold and the load average is here. I'm just going to artificially increase the load. So you can imagine maybe this is like people doing work on the VMs. So they're doing all sorts of computations. And when we have too much computational work, we want to move the VMs so you don't have noisy neighbor problems. Does that make sense? And reschedule them on a different machine. So I'm just going to heat up my poor little laptop here. And watch what happens when we get to 1.5. MGMT should shut down MGMT2 VM. Almost there. Are we going to hit it? There we go. So you see how it disappeared? It shut it down. Now I'm going to just decrease the load by canceling it. It went below. And when it's below for 10 seconds, it will turn it back on. So that's about five seconds, four, three, two, one. And there it goes. Back on. Do you like that? Thank you. So the whole idea is managing things in real time and making these decisions. So you're going to have to shut this off. So I'm almost out of time, so I'm just going to give you some teasers for the future. I've got five minutes plus questions. A few? Yeah, ish. So not too many questions. So there's still a lot of stuff to do. I'm trying to do this as a free software thing. So a lot of things still have to do. There's some bugs. It's not perfect. We have some MGMT contributors here. We've got one here. There's probably some more somewhere. How can you help? You can use this, test it, patch it, share it, document it. Start on GitHub if you want. Blog it, tweet about it if you have Twitter. Discuss it with your friends. Hack on it. Like we're hackers. Like send patches. I left my job at a mostly cool tech company to work on this, which kind of sucks. So I have no money or less money every day. So we need funding. I started a Patreon for fun. So if you want to fund a hacker, funding a hacker is extremely sexy. So you should send me some money. If you want to be sexy, this guy's like, I need some sexy. You can send me money or a patch or tell your friends. If you have a company that wants to get in early and build some of this stuff into the infrastructure, ping me. Let's just recap so far. He's putting the cap back on his pen. I reuse all my jokes because I don't have time to make new ones. Yeah. So I have, there's an IRC, wait, wait, hang on, hang on. I have an IRC channel. There's a Twitter account. We have a mailing list. There's a technical blog of James. You all know about it. So check it out. There's a bunch of blog posts and stuff that you can read about. I'm purple ID on Twitter and IRC and so on. More today. Some crazy person accepted almost all of my FOSTEM talks. So I have another talk later today and another talk even later. I'm going to reuse some of the demos so they're not all brand new. Sunday, I have a container talk and then a five minute lightning talk. And in Ghent, there's this other great conference that Walter helps organize. And we have a whole hackathon and two talks. So if you want to come to Ghent on the sixth, there's going to be a whole day of MGMT hackathon, writing code, writing core stuff, everything you want. And really quickly, if you like this talk, you have to go to the talk page where you see the calendar and there's a secret submit feedback button. So please go over and click on it. This is one of the organizers so if you see him, you can bother him. And lastly, I have some stickers. So if you'd like a sticker, I'll be outside in five minutes. Come grab a sticker if you promise to put it on your laptop so you look cool like me. That's it. Thank you very much. Thank you.
 So I'm a hacker, I work on config management stuff.  I write a technical blog called the Technical Blog of James.  Who's seen it?  Just raise your hand.  Oh wow, if you haven't seen it, just raise your hand anyway so I seem really popular.  Excellent, thank you.  I'm actually a physiologist by training and I do all sorts of DevOps stuff.  So if you want to talk cardiology, let me know.  I used to work on a lot of puppet stuff a long time ago.  Nowadays it seems that every day we're kind of yamling, like Ansible and Kubernetes and  all this stuff.  There's this yaml camp, Twitter account.  And is this what we want?  Is this how you want to build your infra with yaml?  Have you heard of, there's a yaml assembler?  What is going on in the world?  Basically you just, someone writes some code and you blast them yaml at it and hopefully  you get some sort of infrastructure.  I don't know.  Who uses these things?  There's some usefulness but I'm a little skeptical.  And so the real question is, do we want to be yaml programmers?  All right, let's see.  This is my answer to if we want to be yaml programmers.  It's the nope, it's definite nope.  I'm sorry.  YAML is useful for some things but you need the right tool for the right job.  And for building real time distributed systems and infrastructures, I think we really need  a language to be more expressive and also safe.  So I have this software called MGMT that I've been working on.  It's got two parts.  It's got an engine and a language which I'm going to talk about today.  Basically in the engine, you can see the stuff online I've talked about before.  It's kind of like puppet in that there's, sorry, it's kind of like puppet in that you  have a graph of resources that you run through like a dag.  But we can actually run that graph in parallel which is really useful.  It's event driven which I'll show you also.  And it works as a distributed systems.  That's kind of cool.  How's the sound alter?  So do you want to see a quick demo of that?  I'm going to sit down a lot so don't worry if you're scared.  I'm still here.  I'll show you just the classic demo.  Basically I just run MGMT on the left here.  And I just delete all this stuff.  So basically I've asked MGMT to make one file and you can see it's just kind of like puppet.  Creates a file declaratively.  But the cool thing about this is if you remove the file, MGMT is actually running in real  time, it detects this and it puts the file right back.  So you do this, remove the file, comes right back.  And it works so quickly you can really even just do this and cap the file.  And as fast as you go, it's always coming back.  Thank you.  But wait, hold your applause.  So you can actually run this watch command to just ask it to do this as fast as possible  and you can see it's still just going really, really quickly.  So I'm glad you like that.  So pause is awesome, but we do this for every resource.  So files, packages, services, virtual machine, containers, AWS resources and so on.  So it's just a simple example, extrapolates whatever you want to do.  So I want to talk more about the language.  I'm sorry I'm going really fast.  I have a lot of stuff to cover.  And so we actually built a language to describe these systems over time.  Some of the things we want in a language, basically something, the reason why we don't  want YAML, we want something that's safe.  We want it to be very powerful.  So our language is reactive, which lets us do some cool stuff which you'll see.  And we want something that's easy to reason about.  So you want hopefully to write a few lines of very safe code and have that describe a  very complicated infrastructure.  And the reason we want this is because if you make an off by one error or like a YAML  type or whatever, you don't want to blow away a whole data center, right?  Right?  Yes, maybe you do, but if you do, we're going to do it efficiently.  So anyways.  So you want to see some demos?  Come on, you want to do some demos?  All right, thank you.  So here's the first demo I'm going to show you.  I'm just going to start running this over here.  So I'm going to run MGMT on the left.  And basically it's a bit abstract, so I have to apologize for that, but I'm just writing  some code that's going to run.  And the output is just going to build some strings and store it in a text file.  And the reason I'm just storing it in a text file is because that's easy to see what's  happening.  And that text file I'm just going to pull like I did with the watch so you can see what's  happening.  So here's the code.  Oops.  Oops.  Sorry.  One second.  So here's the code.  And I'm just going to run it on the left.  And if you see, I just run this little watch, I've asked it to print out a few things.  So I've asked it to print out the load on the current system.  So load is a function that comes with the language.  We also have date time, which is a built-in function.  And this is a reactive programming language.  So these functions generate a stream of values.  So every second, date time has a new value it wants to produce.  And the language intelligently reevaluates everything that needs to be reevaluated whenever  something changes.  So basically take these values, we put them in this variable, put them all into a big  struct and then print out that struct in this file.  And if you see, we actually can see, is it big enough?  Can you see in the back?  Yeah?  Good.  You can actually see what's happening.  You can see the seconds are incrementing every second.  And you can actually see the load on my machine in real time.  And in fact, just to show you that any sort of data can be used as input, I have this  VU function here.  And this is actually using my microphone in my laptop in real time and recording the room  audio and transforming that into a little graphical representation.  So if I make a lot of noise, you can see it goes up.  So if we're really quiet and then you make a lot of noise, let's see if it works.  Very good.  So I've made this joke before and I've talked about this particular example before.  But imagine you have a server room and you put some microphones around.  And if there's a lot of loud noise, because people are fighting, you could set things  read only for an hour and wait till the fight stops and people settle down.  I mean, these are the sort of things we need to do.  We have to stop responding to pages and monitoring problems.  We need to program the logic in advance and let some sort of engine and language deal with  it.  Does that make sense?  Okay, I want like 300 contributors at the end of this talk.  So that's this demo.  Do you want to see some more demos or have you had enough?  All right, cool.  So it's a big thing to write this big project.  So we just actually landed import and modules.  So you can actually write whole modules of code now, which sounds silly.  It's not great for demos, but just to show you basically the way it works is you have  an import.  So you import a module or another directory or even like an FQDN.  And then all of those things actually give you a separate namespace for each one.  So if you were to import FMT, you could actually run FMT.printf.  So who's ever used Puppet before?  Puppet users.  Puppet had this horrible problem where if you had two modules named SSH, you couldn't  have two.  You'd have to rename one in the code and it was a disaster.  So now you could actually have like, hopefully not, but 10 modules named SSH.  And if you really needed to use both at the same time, you could like import module as  name one and other module as name two.  That sort of thing.  Makes sense?  Silly stuff.  This is sort of like future of MGMT.  So all right, so that's that.  You want to see another demo?  Yeah?  Okay.  You sure?  So I have this really cool demo.  So what I'm going to do, this is a bit complicated.  So first I'm going to be using a runMGMT on my laptop over here.  And what it's going to do, it's going to actually start up a new VM.  Okay?  Let's see if it worked.  And so that's great.  So now I'm going to actually log in to that VM to see what's happening.  So that's the VM on the right side, which is starting up.  Great.  I'm just going to log in so you can see what's happening.  Now you could have it automatically start MGMT on first boot, but I'm not going to do  that because I want you to actually see the log output.  And so I made a little script here just to run MGMT.  So now inside the VM, I'm also running MGMT.  Right?  Normally you would do this automatically.  So MGMT is running.  It's running some different code.  And here's what's actually going to happen.  Oops.  I actually have to kill this for a second.  I'm going to run this inside of screen for a second.  So we can just making a screen session so we can run this twice.  So I'm running MGMT.  And over here in the VM, I'm going to actually run this little script.  It's really small.  Display CPUs, which is just going to run a little loop to print out how many virtual  CPUs are on this VM.  Okay?  So on the right, you can see I have MGMT code which prints out how many CPUs it sees.  And I also have LS CPU which is showing you that there's one virtual CPU.  Makes sense so far?  Okay.  So here's the fun part.  Now it gets exciting.  So the MGMT code on the left, on my laptop, it is actually, there's a reactive variable  which is reading a text file.  And so if I go over here, cat CPU count, it just has a number one in it.  So if I actually echo two to this file, MGMT is going to notice.  It's going to suck in that file into the MGMT code.  It's going to see, oh, the number of CPU, the number is two now.  And then it's going to take that number and set it as the VRT resource CPU count.  And what that's going to do, it's going to hot plug a CPU hopefully.  Now when it does that, inside MGMT, inside the VM, MGMT is actually going to notice  that that CPU count has changed.  And it's going to also use that variable and write out to the file that it has hot plugged  in new CPU.  Does that make sense?  Let's hope it works.  You want to see?  Ready?  Did it work?  I don't know.  Actually, it didn't work.  Oh, sorry.  I cat it.  Sorry.  Wrong thing.  Echo two.  There we go.  There we go.  Cool.  So it changed.  So we can put it to four.  Thank you.  So watch.  Again, over here on the left, MGMT is just waiting and sleeping until that file gets  to change.  You set it to four.  It quickly gets to four.  MGMT inside says it's asking the kernel, hey, how many CPUs are there?  It sees that there's new CPUs and then it runs some code differently.  And just to show you how fast this actually works, I have this little script.  Yeah?  Yeah, thanks.  That's Walter Heck, everyone.  Pull in data.  Check them out.  So I made a little script.  Just a crappy little shell script.  Instead of echoing to that file, just like echoing every time I press a button.  So now we can very quickly press up and down like this and it will change in real time.  So that's the sort of speed.  Now think about this.  Why is this useful?  Like, I'm just doing this as a silly example.  But in real time, imagine you're some VM host.  You have a whole bunch of big servers.  A bunch of them are sleeping.  And then you put all your customers, VMs, all on the same host until the load goes up.  And then you sort of start adding more CPUs to deal with that load, shrinking other CPUs.  Inside the VMs, you can do that as well.  So you can really start programming real time by the second decisions across your whole infrastructure.  Good idea?  Yeah?  Yeah?  Some people are like, I'm not sure.  You'll get there.  So, yeah, any emergency questions about this?  I got more demos if you want to see.  More demos?  All right, cool.  So I'm just going to kill this.  So if I actually shut it down without killing MGMT, it will turn it back on.  So you got to shut this down first.  So I showed you this magic stuff.  So you can hot plug and hot unplug CPUs very, very quickly.  This is all using libvert.  And the CPU count is actually a function in the language that actually lets you decide.  Let's do some scheduling.  How are you, people say it.  So I said that MGMT lets you have a distributed system of different machines.  So let's say you want to do some task on N out of M of the machines.  So what we actually do is we have a language function for that.  It looks like this.  Very simple.  We have this schedule function.  You give it some options as input.  So for example, what kind of schedule you want to use?  Here, this is just a round robin scheduler.  How many machines do you want to use?  How many machines do you want to pick out of the pool?  So here's two.  And I have a TTL of 10.  And then what this function returns, it actually does behind the scenes all this raft distributed system magic.  So you don't have to worry about it.  And it returns a variable that contains which hosts were chosen.  And each host will look through that variable and say, hey, am I one of the hosts in this list?  If so, I'm going to do some stuff.  If not, I wasn't chosen.  Make sense?  So I'm just going to run this.  I'm just going to run a whole bunch of MGMTs on the left here.  I'm just going to show you.  So I'm just going to run this watch.  We can actually do tail dash.  F might be prettier.  Oh, wait.  The tail or what?  Yeah.  So I ran the first one.  And this is basically the output of that first machine.  Again, I'm just asking it, what did you choose as the scheduling decision?  So you want to add a second one.  Just start it up.  And now there's two machines available.  So H1 has decided if you're in H1 or H2, then you should do stuff.  H2 has also decided you're going to be scheduled.  Does that make sense so far?  So every machine that joins the cluster is going to say, I'm part of the cluster.  Do you want to use me?  And the scheduler is going to decide, yes, we're going to use H1 and H2.  And that should be consistent.  If we start a third machine up like this, watch what happens.  This machine starts up.  And you can see it also is part of the cluster.  And it knows that everyone has decided we're picking up two hosts, H1 and H2.  All right.  With me so far.  So watch what happens if I shut down H2.  You see instantly the first machine said, OK, well, now we've got to switch to H3 because we need to pick two hosts.  And there was only two that are available now.  And watch, machines two is actually dead.  That file is just left over.  When I start up H2 again, you can see that it comes back and it joins the existing scheduling decision.  So it doesn't flip flop.  And you can do the same thing.  If you were to kill H3 now, you'll see it'll shut down.  And these machines picked H2.  Make sense?  We'll start up H3 again just to show you one other quick thing.  So if I were actually to forcefully unplug H2, I just forced killing it, you'll see that actually stays at H2 because there's that TTL timeout.  So it's about 10 seconds.  And in a few seconds, you should see it switched H3.  Right.  Perfect.  You get that?  Is that cool?  Yeah.  So what is this useful for?  Anybody have an idea?  So think about this. Let's say you wanted to use some crazy scheduler thing like Kubernetes or whatever.  You don't need to use this huge custom thing.  You can actually just describe what you want to have happen on which machines.  It's just basically a little class which start VMs or start containers or do whatever and have a simple function like this build out your logic.  You don't have to guess what parameters go in a YAML file.  You don't need to know all sorts of fancy stuff.  You describe your infrastructure as code and because it's running in real time in the special language, all that fancy stuff sort of happens automatically.  That make sense?  So you have to use your bit of imagination, but hopefully that's useful.  You want to see more or you had enough?  A little.  It's like a quiet more.  I can do up to two more demos maybe.  All right.  I'm just going to show you another language feature.  Let me just actually kill all these really quickly.  I'm just going to kill them violently.  So I have this quick example.  I showed you in the language about how every value is a sequence over time, but because we have this stream of values, we can actually look back in the past and see what previous values were.  So in this case, we have date time and it just stored in a variable.  And then we can actually take that variable and use curly brackets around it.  So 0, 1, 2, 3, 4 to get the last or previous value and so on.  And you can actually see those values over time.  So I take all those values and print it in a file.  We get the same sort of thing.  I show you the contents.  You get the same sort of thing here where you see the current time and then the time a second ago and the time a second ago over and forever.  So if you use this information, you can actually do some clever things like was the value previously smaller?  Are they going up?  Are they going down?  Is the load going up?  Maybe I should be worried because the slope is really high.  So I want to start up some machines to get ready for that.  Well, quickly, jump in.  You missed the fun parts, but there's more fun.  Hop in.  Foss stem compression.  If there's room, just squeeze over.  So these two kind of things are built into the language and their core primitives.  So why are these useful?  So the reason these are useful is because of this.  I've shown this example before also.  What does this look like, anyone?  It's a thermostat.  I took this photo with my parents' crappy camera in Canada.  You can see it's in Celsius, the correct international units.  And I, yeah, thank you.  I have a clearer photo from the Internet in some sort of weird units.  I don't know what they mean, but just to show you the idea.  And these things, what?  I can't hear you, but a sauna.  Yeah, maybe that's funny.  I make the jokes here.  What's your name?  No MGMT for you.  Okay, so it's a thermostat.  And thermostats have an interesting property.  Does anyone know what it is?  Or has anyone seen my talk before?  I haven't seen it.  What?  Historesis.  Someone pronounced it wrong, but you're all right.  It's hysteresis.  And all that really means is that if you had some sort of threshold,  when you hit that threshold and you turned off the heat  or got to that threshold and turned on the heat,  when you're at that point, you don't want to flick the heat on and off repeatedly.  We just go click, click, click, click, click, click, click, click, click,  and we'll break the thing.  So that's unfortunate.  So hysteresis allows us to actually have a bit of a delay  before we click back on after we go through a number.  So I'm just going to show you a demo of this.  I have a hysteresis example.  So this example, again, it's about ten lines of MGMT code.  What we do is we have this demo  where it's going to start up some VMs  and it's going to start up two VMs  as long as the load on the system is below 1.5.  And when it gets to 1.5, it's going to shut down one of the VMs.  And just to show you all this happening,  you can see the two VMs right here.  I don't know why this guy is still here.  It's just been on my system forever.  And watch what happens when we increase the load above 1.5.  You can see I'm actually printing out that value right here.  So 1.5 is a threshold and the load average is here.  I'm just going to artificially increase the load.  So you can imagine maybe this is like people doing work on the VMs.  So they're doing all sorts of computations.  And when we have too much computational work,  we want to move the VMs so you don't have noisy neighbor problems.  Does that make sense?  And reschedule them on a different machine.  So I'm just going to heat up my poor little laptop here.  And watch what happens when we get to 1.5.  MGMT should shut down MGMT2 VM.  Almost there.  Are we going to hit it? There we go.  So you see how it disappeared?  It shut it down.  Now I'm going to just decrease the load by canceling it.  It went below.  And when it's below for 10 seconds, it will turn it back on.  So that's about five seconds, four, three, two, one.  And there it goes. Back on.  Do you like that?  Thank you.  So the whole idea is managing things in real time  and making these decisions.  So you're going to have to shut this off.  So I'm almost out of time, so I'm just going to give you some teasers for the future.  I've got five minutes plus questions.  A few? Yeah, ish.  So not too many questions.  So there's still a lot of stuff to do.  I'm trying to do this as a free software thing.  So a lot of things still have to do. There's some bugs.  It's not perfect.  We have some MGMT contributors here.  We've got one here.  There's probably some more somewhere.  How can you help? You can use this, test it, patch it, share it, document it.  Start on GitHub if you want.  Blog it, tweet about it if you have Twitter.  Discuss it with your friends.  Hack on it. Like we're hackers. Like send patches.  I left my job at a mostly cool tech company to work on this, which kind of sucks.  So I have no money or less money every day.  So we need funding. I started a Patreon for fun.  So if you want to fund a hacker, funding a hacker is extremely sexy.  So you should send me some money.  If you want to be sexy, this guy's like, I need some sexy.  You can send me money or a patch or tell your friends.  If you have a company that wants to get in early and build some of this stuff into the infrastructure, ping me.  Let's just recap so far.  He's putting the cap back on his pen.  I reuse all my jokes because I don't have time to make new ones.  Yeah.  So I have, there's an IRC, wait, wait, hang on, hang on.  I have an IRC channel. There's a Twitter account.  We have a mailing list.  There's a technical blog of James. You all know about it. So check it out.  There's a bunch of blog posts and stuff that you can read about.  I'm purple ID on Twitter and IRC and so on.  More today. Some crazy person accepted almost all of my FOSTEM talks.  So I have another talk later today and another talk even later.  I'm going to reuse some of the demos so they're not all brand new.  Sunday, I have a container talk and then a five minute lightning talk.  And in Ghent, there's this other great conference that Walter helps organize.  And we have a whole hackathon and two talks.  So if you want to come to Ghent on the sixth, there's going to be a whole day of MGMT hackathon,  writing code, writing core stuff, everything you want.  And really quickly, if you like this talk, you have to go to the talk page where you see the calendar  and there's a secret submit feedback button.  So please go over and click on it.  This is one of the organizers so if you see him, you can bother him.  And lastly, I have some stickers.  So if you'd like a sticker, I'll be outside in five minutes.  Come grab a sticker if you promise to put it on your laptop so you look cool like me.  That's it. Thank you very much.  Thank you.
Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_0_179.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_180_1703.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_1704_1835.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_1836_2543.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_2544_2915.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_2916_4175.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_4176_4259.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_4260_4499.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_4500_4739.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_4740_5759.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_5760_9023.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_9024_10847.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_10848_17795.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_17796_17927.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_17928_18947.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_18948_20423.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_20424_20843.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_20844_21923.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_21924_22019.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_22020_22319.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_22320_22427.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_22428_22499.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_22500_22547.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_22548_23795.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_23796_27839.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_27840_27971.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_27972_30035.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_30036_30179.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_30180_33863.png Mgmt_Config__The_Road_to_0_1_Real-time__autonomous__automation_VzEowt8zWis_33864_33985.png