Using systemd to high level languages
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Geisse, Alvaro Leiva
There is so much more than you can so, than just starting and stooping your service, when you start to interact with systemd from within your application. Lets find out!!! We'll explore how can you interact with systemd from your aplication, by describing how pystemd, a python binding for systemd, was created at facebook, and what type things your programs can take advantage if you just interact with systemd more. From executing section of your code securely to create development environment on the fly, There is so much more than you can so, than just starting and stooping your service, when you start to interact with systemd from within your application.
 I'm Alvaro Leyva, I'm a production engineer at Facebook and Instagram, and I'm here to talk to you about how we use system D in high-level languages. System D is written in C, so that's already a high-level language, but I'm talking like higher-level languages, like Python, Node, API, Node, JavaScript, PHP, all those languages that are not really like nuance and stuff. Okay, cool. So basically, this talk will have like two portions. One is how we as Facebook did a library that talks to system D directly instead of shelling out, and then a few fun things that we do with it. The idea of this presentation is not to say, hey, guys, go and use our library, it's more on the sense like, hey, this is all the things that we learn by creating this library, you should all go and create your own libraries in your own languages. And if you also want to do one in Python, that's the one that I'm going to present, go for it. This is really fun, and this is something that we can all learn from. So let me start with the initial journey on how we start with this. So this talk will start basically when we were at Instagram. We needed to migrate all of our fleet to Cento 7 because that was part of what Facebook was doing. Davida has talked about this for two years now. He promised this that we, he promised in this year that we have already done this, and we were one of the last groups to do this. And the reason why it was hard for us is because we had a lot of maintenance scripts that rely on the underneath service manager. One of the things that we typically do is that we would execute stuff like etc, int, casandra status, and then we parse the output, and then that's how we decided if casandra was running or not running and all the things. So all the things that were not really, it's not that we're not possible, but that was not a good idea to do it with system D. And for good reasons. So for instance, system D comes with system CTL status, which is really great for humans, but it's really hard to parse the output. And system D does provide a lot of toolings to do this from the command line. Even system CTL has a chose status that will kind of ask you individual fields. You can also use bus CTL and all those stuff. But if you are in a high-level language, again, I use this as an enonym for Python, to do this you have to tell out. And at least for my personal experience, telling out, I consider it a little bit clowny on those things. So it would be really nice to have a really nice and native interface to talk to system D. Yeah. So it basically all starts with Dbass, right? So from an external developer, not a system D developer, but from an external developer that wants to interact with system D, this is for me the best decision that could have ever gone into system D, like using Dbass as a way to interface with system D. This basically allows external services to communicate through the Steven and get all the information that you need from it in a way that is not as clowny or hacky as like executing a command, getting a text out, parsing the text, and then if that text is supposed to be like a PIV, convert that into string, sorry, convert that into an integer that has some problems. So it was really kind of a surprise to me when I started working with this that this is also how system D most internal tools work. And it's kind of because we all know that the Dbass daemon is not present at every stage of the machine. Like it's not really present at the beginning and at the end, shut down and start. But for most general things, this is how you communicate, this is how even system D tool communicate. So first of like if you go and read system D code, which I'm assuming that everybody here has done it, you will see all this type of code. Like system CTL communicate to the manager, system D run, which is something that I really like to use and have used, also communicate through Dbass to start the transient units. And even in CTL, like instead of telling out the changing namespace sort of doing other things that you could do, just communicate to the Dbass daemon running inside the machine. I'm pretty sure that I'm going to get corrected about this, but at least this is how I view it from the outside. So Dbass is like really great and it's something that we could really use. In general, like if you forget all about this and you need a really good IPC layer to communicate, like use Dbass. It's really reliable. I would expect people can tell you otherwise. So since everything starts with Dbass, if you're going to interact with system D from a high-level language, you should literally just start with Dbass. So let's see a little bit how Dbass things work. So system D comes with a nice tool that is bus CTL. Bus CTL, when you execute it without any command, it will just show you all the services that are running on your system. And these are all things that you already know since like system D1 or host name that is called the reverse domain. Something. Sorry. I black out on that. Well, basically, these are all the services, right? If you go and explore one of the services, you will see objects. And objects, you can always look at this like name of instantiated attributes or instantiated objects on the system. For instance, for system D, these are all the objects that you can connect. And then you see even though they have like weird names, they all in target or service. So if you don't know anything how this thing works, but you have only work with system D, those doing like start creating units and stuff like that, you will start to see like there is a certain resemblance. And this looks like a really programmatic way to access this data. So if you go and actually do an introspect into this, you will see that it has interfaces that has names that are really similar to what you would expect them to have. Like if I'm inspecting the service unit, you will expect that I will have order.freedestock.systemD.service. And then inside those, you have method and properties. And I'm pretty sure that this is really known for everybody, but at least when I started this, this was all new. So I think it's at least worth mentioning. Method is just a fancy way to say functions. It's something that you will call and will return you something that you can give arguments to it. And properties, again, it's another fancy word for just saying a property is something an attribute that your object has. And it can change through times. I always think of properties like functions that you call without arguments, even though that is not true, but that's a really mental model that I have. So the thing that makes a divas object that you actually, that makes this really powerful are the three interfaces that are the beginning. The first interface is the one that we're going to start using and it was the one that made possible to actually create a library. It was the introspectable that allows you to, without knowing anything, except the address and the object, this will give you all the information that you have to reconstruct the object and actually interface with it. Peer and properties are things that are there and are really cool that we are not going to touch really much, but you can also use it. Okay, cool. So introspect. Introspect is a method of that interface that if you call it, you will get an XML representation of the object and once you have an XML representation of the object, anyone who has lived through the 90s knows that you can construct anything, right? Because so did it, so why can we do it with divas? So in theory, that's all you need to do to work with it. So how do you call it? I'm getting ahead of myself because I'm using a systemd library. There's nothing in my talk so far that talks about systemd. I'm just talking about divas, but I'm getting ahead. So basically, you call that method and then you will get something like this, that is the XML. And you see that it gives me interfaces, a list of interfaces inside you have properties that has the type that it will return. It has the type of a, it has a real name that you will use as a unique identifier for the property. And then you have methods and methods cannot take arguments or they can't take arguments. And then you have arguments with the types and basically the direction. If they're in, it's arguments that you give. You also have the return types that is eventually defined somewhere. So some implementation of divas also returns the name of the argument that is kind of useful. I would prefer that it were there, but if it's not there, it's not a big deal. So we now have a way to construct things. So when we were going to start building this library, we had to find like underneath libraries that we wanted to use. Again, I'm a Python developer if you haven't noticed by the 100 times that I say Python. But so I wanted to use divas Python because it has Python in it and it also has divas. It makes sense. The problem is that that is abiding to live divas that is really old, that is basically written exactly like a month later after the implementation of a specification of divas was written. So it doesn't have all the experience that we have now that we, now this protocol is pretty mature and also the main author of this library set, a disclaimer that says divas Python might not be the best for you to use. So if they also said don't use it, then thank you. I will move on. The next one was Python divas which is also based on Gdivas and it was a really good alternative that I wanted to use. But with that said, I spent a few times reading learner's blog and I find Sdbass as part of the system de-implementation. And this was actually the best thing that we wanted to use. So we wanted to create a library to vint to system de, not to divas. Divas was the protocol. Using this thing will basically give us the same way that system de itself vint to divas and do everything. And also it gives you a little way of cheat in the sense that I could just go to system de and read the code and understand how this do it and then I could implement it on my script in Python and say hey, I created something but no I didn't. For your language, if you want to do it, there is this page over here that has all the bindings for many languages. You should go and again write this in Ruby. Have fun. So the great thing is that, okay, so we want to use Sdbass. There is no Python library for Sdbass. In fact, there is no Python library to vint to live system de. There is not. There are few libraries. None of them did whatever we want. So as all developers do, when they can't find something, they just build it themselves. So we at Facebook created PSTMD. PSTMD is a pun of words because it sounds like system de but also start with py as every Python library should. So yeah, naming things is hard. This was easy. Great. Other options were p1 which basically is p1 but in Spanish. So yeah. We did it. Yeah. So basically we did it because we wanted to bind to Sdbass but we also wanted another word binding to system de we could bind to other libraries that were present like Sddemon and all the things that Davida talked about, that was really cool. So how does this work? We basically Python this library, connects to divas, get the information and you generate classes. Nothing far from the ordinary. I'm pretty sure that everybody can come out in five minutes with code version of what this does. But here is the fun thing that you learn while you are translating basically something that was written in C into Python. Is that there are Cisms or Pythonisms that you need to translate from one language to another. First in C, there is no propagation of exception. So you see this pattern all the time when you read code, you call to a method that returns exit status and side effects on the arguments and then you need to check for the return status to get a unit, sorry, you need to check for the exit status to realize if you need to fail or continue. And then if this method is returned by another method, then that method also needs to check and then there is this propagation that you have to explicitly do. In Python and other languages where you do have exception propagation, all you need to do is check the exit status of the first time that you call the command that actually has this behavior and then you raise. And when you raise the exception, that will bubble up everywhere else. So you don't, if you want to stop when something fails, you don't do anything. You just let the application or the virtual VM does that for you. If you do want to stop, sorry, if you don't want to stop, then you just try to accept your errors. So that's one cool thing that you get and then you're going to see the difference if you ever look to my code that I raise a lot instead of checking. And then I try to accept a lot. The other thing is that systemD at least from my point of view, it's really written in a way that it looks like it's supposed to be object oriented, but it's not because it's not. So you usually give this struct at the beginning of almost all calls. So this really lends to when you are translating this into a language that is object oriented, like heavily object oriented, you get into those things. So you have a class stvus that should be like then you delete the first name and then you just rename all the methods and then they call self. The idea of doing this is that if somebody is read, is used to write Python, this comes really natural. And it's something that's used to write C and wants to write these things, but in Python, because somebody asked him to, it doesn't have to learn so much new things. It just needs to learn that it needs to replace sdvus underscore with sdvus and then addot. That's not that big of a deal. So now how does this look? It's basically you install everything with install.pistending. Then you call it like that and it looks very simple. If you do or.free.stop.systemd1, you will install, sorry, call it from pistending.python and then you call the unit. And then you start the unit in this really pitonic way where you don't have any information about how it is and this will construct everything and then you can start and stop it. And this thing over here encodes the idea of the interface that is completely evaluated when you call for the first time unit. Now, this is, again, you can do it, like if you don't see, you can even do it and see there's nothing magic about this. The great thing is that now for a Python developer or these developers that are high level, that they have never touched C and they also don't want to touch C, you give them like a really nice interface to the stuff that they first didn't know how to do or that they would chill out and then do a really hacky and let's face it, clowny way to get the output. So these things are cool. Also, once you call this method, that it basically automatically calculates the thing that it needs to replace, put it on the bus and everything, you also get the real return value of the thing that in this case would be a job. So then you can verify the job through the bus and everything, right, tight nicely and then you don't have to do any experimentation. I say that a really thing, I said that I really think that system D was really good at making the decision of running Divas. The other thing that I really think system D does really good is giving you output or information about your service and I'm preaching to the choir probably, but when you do system CdL status, you get all this information. And sometimes you actually want this to do in your code, so you do the same thing and then you ask the main PID. And again, the main PID, you get it as an integer because this thing is type of. But then with system D, you can actually do socket activation. You don't do it with socket activation, but you read the information from socket activation when you just get the same headers that you would get by bandwidth with system D. And you do socket from FD and then now a Python developer and maybe I'm selling them short, but that is used to do socket and then socket.list and then socket.asset or bind. Now, in his own language, without changing anything, he can do things. So that's the first part. And now let's talk about fun things that we can do. And a little disclaimer, everything that I'm going to show you, you can already do it with system D run and then bat script and everything. I'm just showing you how you can do things that may not be the best way to do them, but again, you can do it from a high levels perspective. So the first thing that I really like when I start working with system D and the things, I really wanted to have this idea that I could decorate a method, like for instance, this thing over here, that it could be even like a normal code. And then I wanted to decorate and then just run this section of the code in a different secret or even in a different process. So I actually went on and did a few things. This will all be available on the system D page. So you have repository. Right now, they are not because I write them only for this talk. So my idea was that I could decorate this method and then make this method runs without any network access. The idea of this is that sometimes you have to run like non-secure code or code that users provide. So it would be a good idea to just make this thing run in a different C group. And again, this is not the best way to do it, but it's an interesting way of doing it. So yeah. So this is how I saw it. I decorate that. And then what that will make, it will start a unit that it's just a really big no-op. It has remain after exit true. I really know that you don't need to do this, but I just put it as an example. So remain after exit true. So this will execute this. Will not do anything. And then will stay there. And then you set the type one-shot. You set the type delegate to true. And then you set the C group settings that you actually want to preserve. And then you start your settings. Your method. To start your method, I do a little trick that is the following. In a main process, I start this as a child, a fork. The great thing about doing the fork is that I can still hold all the status, all the variables and all the information that I have from the main process. So I have, like, if I declare a global variable, it's available to this method. But I start. I wait for this thing to actually start to have a P. And then I call this little great thing that is called attach process to unit that basically will allow me to make this process, move them to this C group or to this other process. Yeah, something that I said something that makes sense. So then this is what happened in the parent. And then in the children, I start the process. I wait for actually to change the unit and to be running on a different unit. And then I call the methods. What this will do is two things. The first thing is that I can call this. And then when I try to ping, it will not allow me to ping. Even though my application does have this permission, by me changing it to a different C group, it works. This is not the best way to do it, but it really works good for this. The other thing is that now since this is working in a different process, you can literally just do system CTL status, the other process, and then you will see this running. And then this give you all the information that you want. So, yeah, this was one cool thing that we did. I think, okay, never mind. I'm assuming I still have time. So the other thing that I have is just a raise of hand because how many people here are Python developers or coding Python? I love you all, guys. Okay. So I'm not going to purchase the choir. In Python, you have this idea of virtual environments, which allows you to basically set similar but different environment that you have on your system with Python. So you can imagine that there is an analogous concept to the idea of portable services, but having portable Python environment. So instead of you just chipping like a whole system, like a whole distribution, thank you, like a whole distribution, you just chip like your Python code, you just chip the Python executable and then leave Python. And then through the magic of just setting up stuff, you can make it run in a way that is completely isolated from the Python that you have. And this is one example of how we do this. But for instance, you can just start a temporal file system. That's awesome. And then you just pin-mount all the Python that you bring from your application. You just mount it into this environment. So you can mount the, literally, the binary. And then you can mount whatever libraries you want to preserve. Then you do a few configurations and you run. And then it's a good way to actually create virtual environments on the fly in the system D context instead of doing virtual M. So this was actually really cool. With this set, you can do the same thing. It doesn't have to be Python. One other thing that we have tried is that actually instead of doing this is bash. So we can actually make, when a developer, for instance, wants to try, like, they have their own settings that looks really similar to production, but they actually want to try to change interchanging libraries with the ones that they are developing, we change this to bash and then the mounts just change the location where the library should go with the developer one and it works really well. The good thing is that this works in the context of the process and then when you exit the process, it goes out. And again, the mic here is that you create PTY something you've been, seen, STD and STD out. This is the last example that I will give and I think with this, are we running out of time? And I think I really want to have questions at the end. So one typical thing that we usually do as infrastructure people is that we need to, for instance, we have to decommission a whole rack of machines. So for that, if these are databases, you usually need to train them, then take the data, put it somewhere else and run it. And the first thing you do is create a batch script. You execute that batch script but then that batch blocks. So you block the thing and then you cannot actually run more than a few because you don't have resources to run them. So the first thing that developers do, and I've been guilty of this, is that they say, oh, so I'm going to create a service that is going to be listening and then when it gets a call, I'm going to sub-process something created and then this is going to start working and then I'm going to monitor what it's doing and when it's finished, I'm going to call back the main process. And every time that I get that, that's a service manager. Why would you call that if you already are running in Linux with a service manager? So a good way to do that is that instead of doing that, just write your same shell script that you have, do a little bit more interesting stuff, but literally just run it as a transient unit. So this could be like a normal unit and then you just start it, but if you want to create this file on the fly and do all this stuff, you could literally just create a transient unit. You give all the attributes that you want, you can put limitation into your script, change the user, and the good thing is that you should always try to use notify when it's possible. The reason why, well, so this is non-blocking and then you just return the name of the unit that you created and then you just monitor that unit. Now, how do you monitor that unit? One thing that you could do and it's not really useful, but then you could use notify. So notify you say ready equals one, it gives you the status, but what if you can put text into it? And you can. You don't over-avuse this, but in your backup script, you go through your backups and then every time that you make a change, you change the status. If you do that, then if you monitor your service from normal tools, you will always see the change that you did. Then you can actually get it from the text. But that means that your scheduler, what it can do, it can always kind of go and connect to that particular unit and then retrieve the status text. And this is useful when you create interfaces, like human interfaces. Like you allow the script that is doing the backup to communicate how it's doing the things. That way you can read it. That's mainly it. I will not touch that one. Ask me later if you want to. Final thing is that binding to live system this fun, you should all do it. Final language that doesn't have a binding, do it yourself. And that's it. So thank you. And questions. I have one minute. So at least one question or none. Go for it. So I have two questions. The first one is you showed before that there is some auto-generated code out of the debuts, introspection stuff. When is that happening? Like at build time? So in my library, you can do whatever you want. You are free to write whatever you want. It's here. It's like when this with, you create the object and this with in Python world, it's executed a method that is called enter. And that method enter, you get go to system, go to the library, sorry, go to system, go to divas, go to the introspection method, get XML, construct the object and then you return the object that you constructed. The good thing is that these interfaces are really stable and you only need to do these once. We don't do it. We do it every time that you create an object. But you only need to do those once. Okay. And the second question was like if I'm not wrong, there are a few library functions that are asynchronous already insistently. Oh, yes. I'll do you, and that one has like his own loop, I think. Like how do you bridge with the Python asynchronous stuff? So I don't, but I will. So yes, that's the one. So the first thing that I create this library, I create with the specific intention that Python 2 has to be supported, which everybody in the Python world hate me for. And Python 2 doesn't support, I think, I think, like out of the box. So now that we're moving into Python 3 world, it's like I will do, I think, I will make this library completely, I think, if you want to. Like it's not, cool. You're welcome. Cool. Gentlemen in the front that I have never seen. On that slide right there, what is replace? Oh. So that is basically like the start and the stop method. This is like system D from yourself. So I can't give you the better answer than that. But they take arguments on how you want to start, like how do you want the manager to actually start or stop your service. And the most basic one is replace. Go ahead. It's basically if you start a service and this contradicts with another operation that has been queued before, what shall happen? Shall the new operation replace the older ones or shall the new one fail or something like that? So I want, before, if you have a question, before you ask, I want to say, like, this is actually was one of the coolest thing about working in this type of library is that you get to learn, quote unquote, how these things actually work, how the sausage is made, because you get to, like, I call this method doesn't have arguments like what? And then you say, like, oh, there's all this option. There's like three options, I believe. Go ahead. My question was regarding, like, if you got, yeah, on this slide here, like, you're saying that the introspection happens at the time the object is allocated, right? Yeah. So this works for any DBS service? Yes. So how would that look like if I, let's say, talk to, I don't know, a network manager or... So I don't have example for this. Let me see if I can get this one. So basically, this is how the magic work. You, like, I have... Sorry. Okay. So this one, it's like a custom object made understand that it's the concept of unit. But I have an object where this is inherited from that basically allows you to specify any service and any object. When you read that object and you see the XML, I understand that everything that, like, if it is or.free.login.1, I understand that every interface that starts with that, it's like an interface proper of the object. So I exposed it, like, this thing as a proper attribute. So then you have, like, your object that you created.unit, sorry, the user would be this one. And then you would see it, like, it works like that. And then the method works. And the other one that has different names, they are not exposed as separate attributes, but you can always, like, get interface and then give the proper name and then they will have. So what's the work then? Then let's say if I want to talk to network manager or any other DBS interface, what would I do? I would first subclass this generic object thing, fill, like, a handful in of things and then that's it, right? That then the rest happens automatically. Yes, exactly. But in general, like, you don't even have to subclass it because the base class, like, it works on its own. You have to make future calls not necessarily to actually give the name, but you will literally just, if you want to talk, for instance, I do this a lot, like, with machine one, you give this one and then you give the object that would probably be either the machine manager or the machine itself. And then, like you say, everything works like right off the box. Except for stuff like time date, which are, like, really small and they don't really confirm this logic. We can talk about that later. But for things like machine CTL or logging, which are, like, really big classes and they work exactly like system D1, perfect. Okay, sounds great. I'm glad you like it. I don't know if I have time for more questions. I don't have time for more questions, but... Thank you very much.
 I'm Alvaro Leyva, I'm a production engineer at Facebook and Instagram, and I'm here to  talk to you about how we use system D in high-level languages. System D is written in C, so that's  already a high-level language, but I'm talking like higher-level languages, like Python, Node,  API, Node, JavaScript, PHP, all those languages that are not really like nuance and stuff.  Okay, cool. So basically, this talk will have like two portions. One is how we as Facebook  did a library that talks to system D directly instead of shelling out, and then a few fun  things that we do with it. The idea of this presentation is not to say, hey, guys, go  and use our library, it's more on the sense like, hey, this is all the things that we  learn by creating this library, you should all go and create your own libraries in your  own languages. And if you also want to do one in Python, that's the one that I'm going  to present, go for it. This is really fun, and this is something that we can all learn  from. So let me start with the initial journey on how we start with this. So this talk will  start basically when we were at Instagram. We needed to migrate all of our fleet to Cento  7 because that was part of what Facebook was doing. Davida has talked about this for two  years now. He promised this that we, he promised in this year that we have already done this,  and we were one of the last groups to do this. And the reason why it was hard for us is because  we had a lot of maintenance scripts that rely on the underneath service manager. One of  the things that we typically do is that we would execute stuff like etc, int, casandra  status, and then we parse the output, and then that's how we decided if casandra was  running or not running and all the things. So all the things that were not really, it's  not that we're not possible, but that was not a good idea to do it with system D. And  for good reasons. So for instance, system D comes with system CTL status, which is really  great for humans, but it's really hard to parse the output. And system D does provide  a lot of toolings to do this from the command line. Even system CTL has a chose status that  will kind of ask you individual fields. You can also use bus CTL and all those stuff.  But if you are in a high-level language, again, I use this as an enonym for Python, to do this  you have to tell out. And at least for my personal experience, telling out, I consider  it a little bit clowny on those things. So it would be really nice to have a really nice  and native interface to talk to system D. Yeah. So it basically all starts with Dbass,  right? So from an external developer, not a system D developer, but from an external  developer that wants to interact with system D, this is for me the best decision that  could have ever gone into system D, like using Dbass as a way to interface with system D.  This basically allows external services to communicate through the Steven and get all  the information that you need from it in a way that is not as clowny or hacky as like  executing a command, getting a text out, parsing the text, and then if that text is supposed  to be like a PIV, convert that into string, sorry, convert that into an integer that has  some problems. So it was really kind of a surprise to me when I started working with  this that this is also how system D most internal tools work. And it's kind of because we all  know that the Dbass daemon is not present at every stage of the machine. Like it's not  really present at the beginning and at the end, shut down and start. But for most general  things, this is how you communicate, this is how even system D tool communicate. So  first of like if you go and read system D code, which I'm assuming that everybody here  has done it, you will see all this type of code. Like system CTL communicate to the manager,  system D run, which is something that I really like to use and have used, also communicate  through Dbass to start the transient units. And even in CTL, like instead of telling  out the changing namespace sort of doing other things that you could do, just communicate  to the Dbass daemon running inside the machine. I'm pretty sure that I'm going to get corrected  about this, but at least this is how I view it from the outside. So Dbass is like really  great and it's something that we could really use. In general, like if you forget all about  this and you need a really good IPC layer to communicate, like use Dbass. It's really  reliable. I would expect people can tell you otherwise. So since everything starts with  Dbass, if you're going to interact with system D from a high-level language, you should literally  just start with Dbass. So let's see a little bit how Dbass things work. So system D comes  with a nice tool that is bus CTL. Bus CTL, when you execute it without any command, it  will just show you all the services that are running on your system. And these are all  things that you already know since like system D1 or host name that is called the reverse  domain. Something. Sorry. I black out on that. Well, basically, these are all the services,  right? If you go and explore one of the services, you will see objects. And objects, you can  always look at this like name of instantiated attributes or instantiated objects on the system.  For instance, for system D, these are all the objects that you can connect. And then  you see even though they have like weird names, they all in target or service. So if you don't  know anything how this thing works, but you have only work with system D, those doing  like start creating units and stuff like that, you will start to see like there is a certain  resemblance. And this looks like a really programmatic way to access this data. So if  you go and actually do an introspect into this, you will see that it has interfaces that  has names that are really similar to what you would expect them to have. Like if I'm  inspecting the service unit, you will expect that I will have order.freedestock.systemD.service.  And then inside those, you have method and properties. And I'm pretty sure that this  is really known for everybody, but at least when I started this, this was all new. So  I think it's at least worth mentioning. Method is just a fancy way to say functions. It's  something that you will call and will return you something that you can give arguments  to it. And properties, again, it's another fancy word for just saying a property is something  an attribute that your object has. And it can change through times. I always think of  properties like functions that you call without arguments, even though that is not true, but  that's a really mental model that I have. So the thing that makes a divas object that  you actually, that makes this really powerful are the three interfaces that are the beginning.  The first interface is the one that we're going to start using and it was the one that  made possible to actually create a library. It was the introspectable that allows you to,  without knowing anything, except the address and the object, this will give you all the  information that you have to reconstruct the object and actually interface with it. Peer  and properties are things that are there and are really cool that we are not going to touch  really much, but you can also use it. Okay, cool. So introspect. Introspect is a method  of that interface that if you call it, you will get an XML representation of the object  and once you have an XML representation of the object, anyone who has lived through the  90s knows that you can construct anything, right? Because so did it, so why can we do  it with divas? So in theory, that's all you need to do to work with it. So how do you call  it? I'm getting ahead of myself because I'm using a systemd library. There's nothing  in my talk so far that talks about systemd. I'm just talking about divas, but I'm getting  ahead. So basically, you call that method and then you will get something like this,  that is the XML. And you see that it gives me interfaces, a list of interfaces inside  you have properties that has the type that it will return. It has the type of a, it has  a real name that you will use as a unique identifier for the property. And then you  have methods and methods cannot take arguments or they can't take arguments. And then you  have arguments with the types and basically the direction. If they're in, it's arguments  that you give. You also have the return types that is eventually defined somewhere. So some  implementation of divas also returns the name of the argument that is kind of useful. I  would prefer that it were there, but if it's not there, it's not a big deal. So we now  have a way to construct things. So when we were going to start building this library,  we had to find like underneath libraries that we wanted to use. Again, I'm a Python developer  if you haven't noticed by the 100 times that I say Python. But so I wanted to use divas  Python because it has Python in it and it also has divas. It makes sense. The problem  is that that is abiding to live divas that is really old, that is basically written exactly  like a month later after the implementation of a specification of divas was written. So  it doesn't have all the experience that we have now that we, now this protocol is pretty  mature and also the main author of this library set, a disclaimer that says divas Python might  not be the best for you to use. So if they also said don't use it, then thank you. I  will move on. The next one was Python divas which is also based on Gdivas and it was a  really good alternative that I wanted to use. But with that said, I spent a few times reading  learner's blog and I find Sdbass as part of the system de-implementation. And this was  actually the best thing that we wanted to use. So we wanted to create a library to vint to  system de, not to divas. Divas was the protocol. Using this thing will basically give us the  same way that system de itself vint to divas and do everything. And also it gives you a  little way of cheat in the sense that I could just go to system de and read the code and  understand how this do it and then I could implement it on my script in Python and say  hey, I created something but no I didn't. For your language, if you want to do it, there  is this page over here that has all the bindings for many languages. You should go and again  write this in Ruby. Have fun. So the great thing is that, okay, so we want to use Sdbass.  There is no Python library for Sdbass. In fact, there is no Python library to vint to  live system de. There is not. There are few libraries. None of them did whatever we want.  So as all developers do, when they can't find something, they just build it themselves.  So we at Facebook created PSTMD. PSTMD is a pun of words because it sounds like system  de but also start with py as every Python library should. So yeah, naming things is  hard. This was easy. Great. Other options were p1 which basically is p1 but in Spanish.  So yeah. We did it. Yeah. So basically we did it because we wanted to bind to Sdbass  but we also wanted another word binding to system de we could bind to other libraries  that were present like Sddemon and all the things that Davida talked about, that was  really cool. So how does this work? We basically Python this library, connects to divas, get  the information and you generate classes. Nothing far from the ordinary. I'm pretty  sure that everybody can come out in five minutes with code version of what this does. But here  is the fun thing that you learn while you are translating basically something that was  written in C into Python. Is that there are Cisms or Pythonisms that you need to translate  from one language to another. First in C, there is no propagation of exception. So you  see this pattern all the time when you read code, you call to a method that returns exit  status and side effects on the arguments and then you need to check for the return status  to get a unit, sorry, you need to check for the exit status to realize if you need to  fail or continue. And then if this method is returned by another method, then that method  also needs to check and then there is this propagation that you have to explicitly do.  In Python and other languages where you do have exception propagation, all you need to  do is check the exit status of the first time that you call the command that actually has  this behavior and then you raise. And when you raise the exception, that will bubble  up everywhere else. So you don't, if you want to stop when something fails, you don't do  anything. You just let the application or the virtual VM does that for you. If you do want  to stop, sorry, if you don't want to stop, then you just try to accept your errors. So  that's one cool thing that you get and then you're going to see the difference if you  ever look to my code that I raise a lot instead of checking. And then I try to accept a lot.  The other thing is that systemD at least from my point of view, it's really written in a  way that it looks like it's supposed to be object oriented, but it's not because it's  not. So you usually give this struct at the beginning of almost all calls. So this really  lends to when you are translating this into a language that is object oriented, like  heavily object oriented, you get into those things. So you have a class stvus that should  be like then you delete the first name and then you just rename all the methods and then  they call self. The idea of doing this is that if somebody is read, is used to write  Python, this comes really natural. And it's something that's used to write C and wants  to write these things, but in Python, because somebody asked him to, it doesn't have to  learn so much new things. It just needs to learn that it needs to replace sdvus underscore  with sdvus and then addot. That's not that big of a deal. So now how does this look? It's  basically you install everything with install.pistending. Then you call it like that and it looks very  simple. If you do or.free.stop.systemd1, you will install, sorry, call it from pistending.python  and then you call the unit. And then you start the unit in this really pitonic way where  you don't have any information about how it is and this will construct everything and  then you can start and stop it. And this thing over here encodes the idea of the interface  that is completely evaluated when you call for the first time unit. Now, this is, again,  you can do it, like if you don't see, you can even do it and see there's nothing magic  about this. The great thing is that now for a Python developer or these developers that  are high level, that they have never touched C and they also don't want to touch C, you  give them like a really nice interface to the stuff that they first didn't know how  to do or that they would chill out and then do a really hacky and let's face it, clowny  way to get the output. So these things are cool. Also, once you call this method, that  it basically automatically calculates the thing that it needs to replace, put it on  the bus and everything, you also get the real return value of the thing that in this case  would be a job. So then you can verify the job through the bus and everything, right,  tight nicely and then you don't have to do any experimentation. I say that a really  thing, I said that I really think that system D was really good at making the decision of  running Divas. The other thing that I really think system D does really good is giving  you output or information about your service and I'm preaching to the choir probably, but  when you do system CdL status, you get all this information. And sometimes you actually  want this to do in your code, so you do the same thing and then you ask the main PID.  And again, the main PID, you get it as an integer because this thing is type of. But  then with system D, you can actually do socket activation. You don't do it with socket activation,  but you read the information from socket activation when you just get the same headers that you  would get by bandwidth with system D. And you do socket from FD and then now a Python  developer and maybe I'm selling them short, but that is used to do socket and then socket.list  and then socket.asset or bind. Now, in his own language, without changing anything, he  can do things. So that's the first part. And now let's talk about fun things that we can  do. And a little disclaimer, everything that I'm going to show you, you can already do  it with system D run and then bat script and everything. I'm just showing you how you can  do things that may not be the best way to do them, but again, you can do it from a high  levels perspective. So the first thing that I really like when I start working with system  D and the things, I really wanted to have this idea that I could decorate a method, like  for instance, this thing over here, that it could be even like a normal code. And then  I wanted to decorate and then just run this section of the code in a different secret  or even in a different process. So I actually went on and did a few things. This will all  be available on the system D page. So you have repository. Right now, they are not because  I write them only for this talk. So my idea was that I could decorate this method and  then make this method runs without any network access. The idea of this is that sometimes  you have to run like non-secure code or code that users provide. So it would be a good  idea to just make this thing run in a different C group. And again, this is not the best way  to do it, but it's an interesting way of doing it. So yeah. So this is how I saw it. I decorate  that. And then what that will make, it will start a unit that it's just a really big no-op.  It has remain after exit true. I really know that you don't need to do this, but I just  put it as an example. So remain after exit true. So this will execute this. Will not  do anything. And then will stay there. And then you set the type one-shot. You set the  type delegate to true. And then you set the C group settings that you actually want to  preserve. And then you start your settings. Your method. To start your method, I do a  little trick that is the following. In a main process, I start this as a child, a fork.  The great thing about doing the fork is that I can still hold all the status, all the variables  and all the information that I have from the main process. So I have, like, if I declare  a global variable, it's available to this method. But I start. I wait for this thing  to actually start to have a P. And then I call this little great thing that is called  attach process to unit that basically will allow me to make this process, move them to  this C group or to this other process. Yeah, something that I said something that makes  sense. So then this is what happened in the parent. And then in the children, I start  the process. I wait for actually to change the unit and to be running on a different  unit. And then I call the methods. What this will do is two things. The first thing is  that I can call this. And then when I try to ping, it will not allow me to ping. Even  though my application does have this permission, by me changing it to a different C group, it  works. This is not the best way to do it, but it really works good for this. The other  thing is that now since this is working in a different process, you can literally just  do system CTL status, the other process, and then you will see this running. And then this  give you all the information that you want. So, yeah, this was one cool thing that we  did. I think, okay, never mind. I'm assuming I still have time. So the other thing that  I have is just a raise of hand because how many people here are Python developers or  coding Python? I love you all, guys. Okay. So I'm not going to purchase the choir. In  Python, you have this idea of virtual environments, which allows you to basically set similar  but different environment that you have on your system with Python. So you can imagine  that there is an analogous concept to the idea of portable services, but having portable  Python environment. So instead of you just chipping like a whole system, like a whole  distribution, thank you, like a whole distribution, you just chip like your Python code, you just  chip the Python executable and then leave Python. And then through the magic of just  setting up stuff, you can make it run in a way that is completely isolated from the  Python that you have. And this is one example of how we do this. But for instance, you can  just start a temporal file system. That's awesome. And then you just pin-mount all the  Python that you bring from your application. You just mount it into this environment. So  you can mount the, literally, the binary. And then you can mount whatever libraries  you want to preserve. Then you do a few configurations and you run. And then it's a good way to actually  create virtual environments on the fly in the system D context instead of doing virtual  M. So this was actually really cool. With this set, you can do the same thing. It doesn't  have to be Python. One other thing that we have tried is that actually instead of doing  this is bash. So we can actually make, when a developer, for instance, wants to try, like,  they have their own settings that looks really similar to production, but they actually want  to try to change interchanging libraries with the ones that they are developing, we change  this to bash and then the mounts just change the location where the library should go with  the developer one and it works really well. The good thing is that this works in the context  of the process and then when you exit the process, it goes out. And again, the mic here  is that you create PTY something you've been, seen, STD and STD out. This is the last example  that I will give and I think with this, are we running out of time? And I think I really  want to have questions at the end. So one typical thing that we usually do as infrastructure  people is that we need to, for instance, we have to decommission a whole rack of machines.  So for that, if these are databases, you usually need to train them, then take the data, put  it somewhere else and run it. And the first thing you do is create a batch script. You  execute that batch script but then that batch blocks. So you block the thing and then you  cannot actually run more than a few because you don't have resources to run them. So the  first thing that developers do, and I've been guilty of this, is that they say, oh, so I'm  going to create a service that is going to be listening and then when it gets a call,  I'm going to sub-process something created and then this is going to start working and  then I'm going to monitor what it's doing and when it's finished, I'm going to call  back the main process. And every time that I get that, that's a service manager. Why  would you call that if you already are running in Linux with a service manager? So a good  way to do that is that instead of doing that, just write your same shell script that you  have, do a little bit more interesting stuff, but literally just run it as a transient unit.  So this could be like a normal unit and then you just start it, but if you want to create  this file on the fly and do all this stuff, you could literally just create a transient  unit. You give all the attributes that you want, you can put limitation into your script,  change the user, and the good thing is that you should always try to use notify when it's  possible. The reason why, well, so this is non-blocking and then you just return the  name of the unit that you created and then you just monitor that unit. Now, how do you  monitor that unit? One thing that you could do and it's not really useful, but then you  could use notify. So notify you say ready equals one, it gives you the status, but what if  you can put text into it? And you can. You don't over-avuse this, but in your backup  script, you go through your backups and then every time that you make a change, you change  the status. If you do that, then if you monitor your service from normal tools, you will always  see the change that you did. Then you can actually get it from the text. But that means  that your scheduler, what it can do, it can always kind of go and connect to that particular  unit and then retrieve the status text. And this is useful when you create interfaces,  like human interfaces. Like you allow the script that is doing the backup to communicate  how it's doing the things. That way you can read it. That's mainly it. I will not touch  that one. Ask me later if you want to. Final thing is that binding to live system this  fun, you should all do it. Final language that doesn't have a binding, do it yourself.  And that's it. So thank you. And questions. I have one minute. So at least one question  or none. Go for it.  So I have two questions. The first one is you showed before that there is some auto-generated  code out of the debuts, introspection stuff. When is that happening? Like at build time?  So in my library, you can do whatever you want. You are free to write whatever you want.  It's here. It's like when this with, you create the object and this with in Python world, it's  executed a method that is called enter. And that method enter, you get go to system, go  to the library, sorry, go to system, go to divas, go to the introspection method, get  XML, construct the object and then you return the object that you constructed. The good  thing is that these interfaces are really stable and you only need to do these once.  We don't do it. We do it every time that you create an object. But you only need to do  those once.  Okay. And the second question was like if I'm not wrong, there are a few library functions  that are asynchronous already insistently.  Oh, yes.  I'll do you, and that one has like his own loop, I think. Like how do you bridge with  the Python asynchronous stuff?  So I don't, but I will. So yes, that's the one. So the first thing that I create this  library, I create with the specific intention that Python 2 has to be supported, which everybody  in the Python world hate me for. And Python 2 doesn't support, I think, I think, like  out of the box.  So now that we're moving into Python 3 world, it's like I will do, I think, I will make  this library completely, I think, if you want to. Like it's not, cool.  You're welcome.  Cool.  Gentlemen in the front that I have never seen.  On that slide right there, what is replace?  Oh. So that is basically like the start and the stop method. This is like system D from  yourself. So I can't give you the better answer than that. But they take arguments on how  you want to start, like how do you want the manager to actually start or stop your service.  And the most basic one is replace. Go ahead.  It's basically if you start a service and this contradicts with another operation that  has been queued before, what shall happen? Shall the new operation replace the older  ones or shall the new one fail or something like that?  So I want, before, if you have a question, before you ask, I want to say, like, this  is actually was one of the coolest thing about working in this type of library is that you  get to learn, quote unquote, how these things actually work, how the sausage is made, because  you get to, like, I call this method doesn't have arguments like what? And then you say,  like, oh, there's all this option. There's like three options, I believe. Go ahead.  My question was regarding, like, if you got, yeah, on this slide here, like, you're saying  that the introspection happens at the time the object is allocated, right?  Yeah. So this works for any DBS service?  Yes. So how would that look like if I, let's say, talk to, I don't know, a network manager  or...  So I don't have example for this. Let me see if I can get this one. So basically, this  is how the magic work. You, like, I have... Sorry. Okay. So this one, it's like a custom  object made understand that it's the concept of unit. But I have an object where this is  inherited from that basically allows you to specify any service and any object. When  you read that object and you see the XML, I understand that everything that, like, if  it is or.free.login.1, I understand that every interface that starts with that, it's like  an interface proper of the object. So I exposed it, like, this thing as a proper attribute.  So then you have, like, your object that you created.unit, sorry, the user would be this  one. And then you would see it, like, it works like that. And then the method works. And  the other one that has different names, they are not exposed as separate attributes, but  you can always, like, get interface and then give the proper name and then they will have.  So what's the work then? Then let's say if I want to talk to network manager or any other  DBS interface, what would I do? I would first subclass this generic object thing, fill,  like, a handful in of things and then that's it, right? That then the rest happens automatically.  Yes, exactly. But in general, like, you don't even have to subclass it because the base class,  like, it works on its own. You have to make future calls not necessarily to actually give  the name, but you will literally just, if you want to talk, for instance, I do this  a lot, like, with machine one, you give this one and then you give the object that would  probably be either the machine manager or the machine itself. And then, like you say,  everything works like right off the box. Except for stuff like time date, which are, like,  really small and they don't really confirm this logic. We can talk about that later. But  for things like machine CTL or logging, which are, like, really big classes and they work  exactly like system D1, perfect.  Okay, sounds great.  I'm glad you like it. I don't know if I have time for more questions. I don't have time  for more questions, but...  Thank you very much.
Using_systemd_to_high_level_languages_lBQgMGPxqNo_0_1163.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_1164_2039.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_2040_3899.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_3900_11015.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_11016_11243.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_11244_11435.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_11436_11747.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_11748_13175.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_13176_13967.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_13968_14747.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_14748_15419.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_15420_18935.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_18936_19571.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_19572_20447.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_20448_20699.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_20700_20927.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_20928_21047.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_21048_21167.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_21168_21395.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_21396_21479.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_21480_24491.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_24492_24899.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_24900_25811.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_25812_25943.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_25944_26303.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_26304_29975.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_29976_30275.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_30276_30515.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_30516_34463.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_34464_35207.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_35208_35879.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_35880_35927.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_35928_36179.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_36180_36527.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_36528_37259.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_37260_37823.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_37824_39479.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_39480_39935.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_39936_44003.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_44004_44303.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_44304_44399.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_44400_46103.png Using_systemd_to_high_level_languages_lBQgMGPxqNo_46104_47213.png