Maximizing Throughput on Multicore systems with Erlang
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Guberman, Irina
Erlang is the best language there is for getting the most out of your concurrent system - it was created with multi-core processing in mind! However, to get the best of it we need to understand what our concurrent processes are actually doing and how they'll play out with our system's physical limitations, a concept also known as mechanical sympathy. I would like to explore this subject based on my project at Ubiquiti Networks and take a look at an existing tool that could make the task of maximizing throughput on you multicore system easy and fun: the "Jobs" Framework by Ulf Wiger. Talk objectives 1. Explore a business case scenario of a highly concurrent HTTP server requiring high CPU utilization. 2. Explore "Jobs" framework in great detail with code snippets and fun pics. By the end of it anyone can start using it even developers with no Erlang background! 3. And last but not least: Demonstrate that Erlang is a fantastic choice for writing highly concurrent applications!
 Hello, everybody. My name is Irina. I hope you're ready to get started. I'll be talking about maximizing throughput on multi-core systems with Erlang. I just wanted to get a feel for how many people here know a little about Erlang. And how many people are seriously considering learning Erlang. All right. So, yes, then, let's get started. So, what we're going to, what I'm going to talk about is an actual project I was working on at Ubiquiti Networks. That's my previous company. And we were working on a Rust-WHTP server handling audio configurations. So, we're going to go on a deep dive on what we had to deal with and how we solved the problems that we run into with Erlang. So, what we have is audio configurations coming from various audio devices or a group of devices or a whole private branch exchange. And that's how the audio configuration looks very simple. It has a text audio greeting and then the audio file output formats. And that's what our request will be. And we get multiple such requests. So, when our server gets this request, we're going to be handling the usual Rustful operations. These are not something I want to talk about. They're not that interesting, but we're going to be talking about the fun part, the actual audio conversions. So, the text that we get will be converted to an audio file. And then resulting audio file will be converted to other multiple formats that we have in the request. And then once we get the audio converted, we need to save it to Amazon Simple Storage Service, also known as S3. And as the audios are saved to S3, we're going to save the URL in our database so that the clients know that they are ready to be retrieved. So, that's pretty much the flow of our request handling, that's our request handling flow. So, a couple of things we needed to worry about is that audio conversions are very CPU intensive. And saves to S3, on the other hand, are easy on the CPU, but might take an unpredictably long time. So, obviously, with this kind of setup, we can have our clients waiting to get their request back immediately. So, what we did was validate the request and send a reply back, synchronously send a reply back to the client saying that the request is processing and then process the rest of it asynchronously. So, let's take a look at our approach in airline. So, for every request that came in, we created a separate financial state machine, a separate process, a separate financial state machine. So, let's say we have audio ID 101 coming in with this request and we spawn a financial state machine to process that one request. And we are going to give this financial state machine ID equal to the audio ID and we'll see in just a second why we do that. So, we have another request, so the financial state machine is processing our audio request. Now, another audio request comes in, we spawn another financial state machine. So, this is ID 100 and this is FSM with ID 100 as well. Now, audio ID 104 comes in, we spawn FSM with ID 104 and it's running. And finally, another audio ID 104 request comes in and that's bad because, I mean not bad, but it would have been bad, but because all sorts of race conditions and problems could arise from that if you're processing the same audio at the same time, but we know we are running this, the FSM was ID 104 is already running, so this request will just be denied and it won't pass the validation. So, that's why we actually keep tracks of the IDs of the FSM we are running to make sure that we don't allow any simultaneous requests for the same audio. So, we're going to have more of this spawn on the screen and I know it's kind of tiring, I'm bringing up these FSMs on the screen, but just imagine how tired our server is, it just keeps getting those requests and spawning them and it's going to be hundreds and thousands of them. 101 just finished, great. Oh, and we got another 101 request coming in. And now it's okay with us spawning another 101 of FSM because the other one finished, so it's totally okay to process another update. That's a lot of processes we just started, but that's, like I said, this is just a small example, we are going to have hundreds and thousands, whatever number of HTTP requests are coming in, for every single one of them, whatever the number that number is, we're going to spawn a new process to just dedicate, dedicated process to that one request. And that might sound scary to people who are not coming from airline world, but it's perfectly okay for an airline developer to spawn as many processes as they want because airline processes are very cheap and they're actually very fast to start. So, you can spawn hundreds and thousands of them, it's perfectly fine. It's the airline VM that's going to take care of properly allocating resources to the underlying OS strats. And you as an airline developer, you don't have to worry about concurrency and multithreading at all. You really just have to worry about your business case and if you need to spawn a process, just go for it, it's perfectly fine. A couple of other things that I wanted to show you how, a couple of other considerations was implementing things that I showed you on a previous screen. So, we had our FSMs as we were spawning them, we were assigning them an ID. So, when you spawn an airline process, it will have a process ID associated with it, but that doesn't help us, it's just some random numbers. We wanted to have an alias that same idea as the audio ID in our particular case. So, what we're going to do there is a concept known as process registry in our wing and there are some built-in functions to register a process. So, if you want to locally register a process within Pacific alias, you can call register name and process ID. Or if you want to register your process globally, so it can be discoverable across multiple nodes, you can call, there is a module global, it comes with airline distribution and you can call global register name. There is a little bit of a problem for us, was this to built-in functions. They only accept atoms as process names and atoms are just constants with a name. So, the problem with atoms is that atom table is never garbage collected. Then just think about it, why would you garbage collect constants? So, in our particular case, these methods are not going to work because we will have to convert those audio IDs to an atom and it's actually possible to do this in our line, but it's really not advisable. You don't want to dynamically generate atoms in our line because you're going to eventually have a serious memory leak in your node. So, let's talk about how stable, how reliable airline is and how airline nodes are running for years. It's very true, but if you do silly things like that, you're just going to run out of memory. So, it's a huge node generating atoms dynamically in airline. So, that won't work for us. This is a warning. If we need to do process registry and we have actually created processes dynamically, we got to use something else. Luckily, there are other libraries in airline and they're quite mature that we can use. One of them is Gprog written by Ulf Wigger. His name we're going to hear one more time today just a little later. It's a great library, but one important thing about it is that we can use any term as a process alias. It actually has seven other bullet points of goodness, but we don't care about them at the moment. The actual library we use is called CPG, Clarity Process Groups. If you look at the read me, you will see the author says that it's very different in the implementation comparing to Gprog, but bottom line is you can use any term as a process alias. So, that's all we care about. It actually has some other nice things about it. You can read the read me if you care about the specific libraries. Of course, write your own library is also an option. So, if you do that, there is only one thing you need to remember. You should be able to use any term as a process alias or at least a non-atom. I'm going to go over a little bit, dive a little more into that problem. You're not going to be always calling this method explicitly. If you use generic OTP behaviors, OTP, I don't know if people know, I'll just briefly say that underlying OTP is a framework that takes care of all the generic behaviors so you don't have to reinvent the wheel. If you want to start a server, you just use gen server and implement all the callbacks and you don't have to worry about the boilerplate code. So, in our case, it's actually gen FSM. It's also one of the constructs that comes with modules that comes with underlying OTP. So, one of the callbacks is called start link. That's being called when you actually start your server. If you start with a server name, the server name can be of three different types, local name. If you give the server name local name, the local register method we just saw will be called implicitly. So, you're not going to call it, but it's going to be called for you under the covers. Then, if you give it a global name, then the global register name that we just saw will be called under the covers. So that's bad because we're going to end up with, if we just put our audio ID, which is an integer, or some kind of string into this method, it's just going to crash with bad argument. And so, this won't work for us. That's just a warning that we shouldn't be using this. But there is a third option. That one actually could work for us if we use the proper module. One of those modules that we saw that actually use, that can use any term or at least an atom term to register a process. So, we can, if we implement our own module to do process registry, we need to implement these four methods and kind of similar to how a global module does it. Next interesting thing to take a look at is supervisors, Erlang supervisor strategies. One of, this is also a very important concept in Erlang, supervision tree. That's what makes Erlang so robust. If a process goes down, we always create, of course, it's up to us as Erlang developers to create a supervision tree and make sure that the processes that are spawned are supervised. But that's pretty much what Erlang developers always do. They have everything, every process you create will be supervised and you kind of design your supervisor strategies. Then we are going to look at some supervisor strategies that Erlang provides. So, first one is one for one. The green thing on top is the supervisor. The yellow things are the running processes. So, we have, if one, so one for one, one for one process crashes, that one process is being restarted. The next supervisor strategy is one for all. If one process is crashed, all of the children of that supervisor are being restarted. You could think of scenarios where you can use, where this can be used, possibly that all of them are somewhat interrelated. So you need to restart all of them. The next one is rest for one. It's one process crashes. It gets restarted, but then all the children also depend on that child will be restarted as well. So it's a little more complicated. These are pictures. These are not my drawings. These are drawings by Fred Hubbard. He is the author of a really awesome book, Learn You Some Erlang. If you are interested in learning Erlang, this is a perfect book to read. It's very funny and it goes into dupes and it's going to alert you about many different things you should know before you start your production, like before you start your production, writing your production code. So there is a fourth type of a supervisor. It doesn't have a picture. It's very similar to one for one. It's called simple one for one. It is not simple at all. It's actually very, very different from the other three. The other three supervisors are, let's call them static supervisors. When a supervisor is created, its children are also created and it knows what it's supervising. This dynamic, simple one for one supervisor is very different. It's kind of like if you have a startup company and you know you need to hire many developers, but first thing you do is hire a supervisor and then you tell him, hey, you don't have anybody to supervise right now because we are still hiring them, but you're going to be supervising our developers. So that's how simple one for one supervisor works. So we're going to create it. We're going to tell it, you're going to be supervising this audio handlers, but it's not going to have any processes to supervise at first. And I was going to, I was going to like ask you which one of the strategies should we be using, but I already think give it out. So obviously it's going to be simple one for one supervisor. So just in general, if you have a situation when you are creating a, when you're creating, spawning a separate process for every request and you don't know how many processes like that you'll be creating. Simple one for one supervisor is probably a good idea. So next consideration is audio conversions. So what are we going to do about that? Audio conversions are not a novel problem. Luckily we already have, there are already C executables that they care a little bit for us. So for text to voice conversions, we use substrel service. There is such a thing. If anyone, if you've never done anything like that, you can use substrel. You can just play with it and have some text converted to voice and different funny voices. And it's, you can just play with that service. And the other service is the one that we used for converting the resulting way file to other formats is called socks. So just basically existing executables we use. However, if your situation is not as simple and you actually need to roll your own, like write your own processing, the, the recommendation would be for airline developers to write maybe a single unit of work and see rather than in the airline. And the reason is if you need to, if you need to manipulate large data structures like huge arrays or huge matrices and like, this is one thing you don't want to do in airline because airline data structures are immutable. So just imagine the amount of CPU and memory that will go into copying all this, when you make slight modifications to them. So it's better to write those in single, like a single unit of work and see and then let airline manage the, manage the concurrency of it. And I would like to quote my friend and big airline activist, Garrett Smith, he's a Chicago airline meetup organizer, think of airline as a manager, not a worker. Next problem is saves to S3. So let's say we are this busy FSM, we are dealing with all the coordinating saves to the database and coordinating audio conversions. Now we got, we created the S3 URL that we need and then we created, we have the audio binary that we need to save. And, sorry. So what we are going to do, we are not going to just sitting there waiting for, we are not just going to, we can't afford to just send this request to S3 and sit there waiting for it. What we are going to do is spawn a child process and this is the code to do this. This is how we are going to send the request to S3. So we are going to spawn a child process that is going to do nothing but just sit there waiting for that request to complete and we can just go on doing our other tasks. Important thing to do is provide this process with our own process ID. That self method is our process ID and we are going to send, give it to that request so it knows where to send the reply back. So this is the process code. So it sends the S3 request, just waits for it to complete and then when it gets the reply, it sends the result back to us. The FSM here is the process ID is our own process. If we are the FSM, this would be our process ID. And once it's done, it sees us to exist. So again, we solve this problem by just spawning a process. If you are an underlying developer, you can just spawn a process if you need something to process the synchronously. You really don't have to worry much about it. Next thing, so audio conversions as I mentioned before, they are very CPU intensive. So what's going to happen if we are spawning thousands of finite state machines and those finite state machines will be spawning, like we'll be running audio conversions, thousands of them simultaneously. As powerful as our language is, we are properly doing concurrency for us. We only have that much CPU power. So if you don't think what can our machine do at the same time, it won't do us any good. So what's going to happen is those audio conversions will heat up all of the processing power and we have absolutely no room at some points in time. We'll have absolutely no room for other important things, like say, is to the database actually accepting HTTP requests. So, I'm going to end up in a pretty ugly situation, commonly known as overload. So what can we do? Well, maybe an easy answer. We can just limit simultaneous audio conversion jobs to number of CPUs minus one and reserve at least one CPU for other high priority work. So how can we do this? We can maybe write our own code to do it. It sounds pretty simple, right? So maybe we can write our own code to do this. But we are pretty busy. We have a lot of business requirements to take care of. So what we did was we used an existing library. And the library we chose, the jobs framework by Ulf Wigger. And the rest of this talk I will actually introduce you to this library and the point of me doing this is not necessarily for you to learn how to use this library, but more to understand what it takes to write a library, kind of what kind of things you need to worry about when you are writing your own library or when you are actually kind of when you are implementing overload prevention in your code, what kind of things might come up in the process. So what this framework is, is a job schedule for load regulation. And its sole purpose in life is just overload prevention. So first important, first important goodie that this framework gives us is, oh, I'm sorry, I just one thing before I go into that. This framework was the inspiration behind this jobs framework is described fully in this article that Ulf Wigger wrote in August of 2010. And it's actually a very interesting and serious article on overload prevention. You might want to consider reading it. But I'm just showing you this article just because I want to show you this diagram. This diagram we see the failures in the US public switch telephony network out each minutes by cause. And as obviously we see here that overload is the leading cause and failures. So I hope this diagram is a good proof for you to consider overload prevention in your code before it goes into production. So now let's take a look at what a job framework has to offer us. So first, that's what we actually use the counter based regulation the way it works. You set up how many jobs of the certain kind at the same time you want to run. And only that many jobs will in the jobs framework will make sure only that many jobs are running. Anything else gets queued up and only starts executing when the other jobs are done. So I kind of have a little visual demonstration of how it works here. And what I would like to do next is kind of deep dive into how we set it up with the jobs framework. This is a little more, a little more airline code here, but just kind of to show you how what it takes to set something like that was airline. So first thing we need to do is add the jobs repository link to the job, rebar config dependencies. Rebar is the build tool in airline. And you, the rebar config is the, it's configuration and that's how it's going to look. Step two, we would need to set up the actual jobs configuration. So we create this audio conversion skew for our specific case and you assign it a regulator called counter and we give it a limit of seven. Let's say we have eight CPUs. So we decide to give it a limit of seven. Of course, that's the number that you need to think about it depending on what kind of hardware you'll be running your process on. And now this is the vast complexity of our business logic. And anywhere within that vast complexity to get your audio conversions regulated, all I had to do is just drop the call to audio conversions in a call to the job server. And that's it. Our audio conversions are counter regulated. Very simple. And I also am going to show you my real code. So this is my code. And right where I needed to do the text to voice conversion, I just dropped it in the call to the job server and it doesn't matter where you do it in your code, you wrap it in the call to the job server. It's going to be, it's going to make sure that no more than seven jobs like that are running. That's, I think it's pretty cool. So one last stop, we actually have been mentioning the job server. So we need to actually have it running. So in order to have that job server or application running, we need to add it to our application startup list. And now you might be thinking, hey, she's, she made it sound so easy, but we actually need to have that job server running. So I need a set extra server running in my machine for this whole magic to work. Well, as an airline developer, you don't have to worry about starting an extra service on your machine because really, airline is pretty, is kind of like an, it's a pretty unique thing actually. It's like an operating system for your code. Just think about it. So you can start any servers on your machine as you want. You don't have to talk to your manager or your operations team manager about, oh, I need to this, nobody has to maintain it. It's really up to you what other services you start with your node. So we need the job server running, but that's not a problem at all. And it's actually, it doesn't have that much overhead either. I think it's more like, it is there, but it's, it's overhead is totally worse. The overload, it prevents you that it gives us, it's totally worse. So the next, next great thing that job framework gives us, before we look at it, I wanted to show you this little video. So let's pretend that these two ladies are our HTTP server and the chocolates that are coming in our HTTP requests. So they're coming in at a very nice steady pace and they can easily handle them. And hopefully that's how our server is going to operate all the time. But no, look at that. That starts, this request is starting to go faster and faster and it's not looking good anymore. Oh my gosh. And just think about it. This could be your HTTP server in production. And this could be your HTTP server in production in the middle of the night. And you probably don't like that. So in order to prevent this kind of situation, the job framework gives us rate based regulation. So again, a little visual demonstration of what rate based regulation is. The request might come in at whatever pace and you just get, but they'll get through the server at a very specific pace that you allow. So given a frequency F, it ensures that the rate of accepted jobs does not exceed F. So to set up the, to set up the, so to set up our HTTP request running at a specific pace, we're going to create a new queue called HTTP requests and we're going to give it a rate, a regulator name rate and we give it a limit of 1000 requests per second. Okay. That number is of course just out of the blue. You need to measure your server performance and decide yourself what is the best rates your server can handle. So that number, you need to really worry about it. Of course, it's much better to worry about it before everything's in production than afterwards. Couple of, so now I showed you two great things that the job framework brings, but I personally had some what ifs creeping into my head. What ifs, we get a denial of service attack. Well, I guess it's okay because you know, we have this rate regulation set up with our HTTP server. So no matter how many requests are pounding our server, they're going to still be going through our server at a specific pace, right? Well, unfortunately no, because all the requests that didn't make it will be actually queuing up and that accumulating in the queue and that queue eventually will fade and blow our server's memory. So what can we do? Job framework took care of that. We can limit our queue size and this is how we do this. We basically just give it a max size. So I'm just trying to show you here a very nice, sort of a very nice API. It's very easy and you can address very numerous concerns and you can just add, so you just have to add max size in your queue and that will hopefully save you from this scenario. What if we have requests that have a strict timeout? Why would we keep them in the queue? So let's say I used to work in an artifact company and we had, the RTB requests were, had various real time bidding providers would give us very strict timeouts of like no more than 200 milliseconds. So if your request has been sitting in the queue for longer than that, why would you keep it there? So to deal with that, we need to limit the request playtime in the queue. There is no point of keeping them around if they're no longer relevant. So job framework has answered to that. Great. We can set the max time of a request in the queue. We can give it, let's say 200 milliseconds. 200 milliseconds is the default unit. So we'll create a new queue called RTB requests and we're going to give it next timeout of 200. What if we also have some long running requests like payment processing requests? Those, we don't want a timeout. We can let them run for five minutes or 10 minutes. We want to make sure they come to, they actually complete. But even, even such requests will have a reasonable timeout because if they're running, if they've been running for 10 minutes, probably something's wrong. So no matter how important and long running your request might be, it's, it's still, it still might have a reasonable timeout. So let's create a queue for those types of requests. Let's call them payment requests queue and we'll give it a max time of 10 minutes. So now we have two types, we actually ended up creating two queues to be able to differentiate them by the max timeout. And what if we still want our requests, HTTP requests coming in at 1000 a second, regardless of what kind of, whether it's a RTB request or a payment request. By the way, these are very extreme cases. There is no way you're going to have a server that processes this kind of requests. You probably have two different servers doing it, but this is just an example. I made, I took some extreme cases, just an example, not real, not a real life example. So for that scenario, the job framework has yet another type of regulation. It's called group rate regulation. Again, this is a visual example of what it does. It will, I don't know if it helps, but I tried. So the group rate regulator allows us to set a cap on how many jobs from a group of queues can be executed per second. So this is, the setup gets a little more complicated now. We create a group rate regulator with a limit of 1000. We'll call it HTTP request rate. And now we assign this group rate regulator to our RTB request. And then we assign this same group rate regulator to our payment request. So now all our requests, regardless of their type, will be paced at 1000 per second. Now audio conversions, we haven't talked about those in a while. What if suddenly something else is happening in our machine and our CPUs are busier than normal and the CPU minus one, which we thought would work for us, is no longer working? The answer to that is not as trivial as before, but the jobs framework has an answer to that nonetheless. It comes with sampler framework. And sampler framework is, what it does, it actually sends feedback about various aspects of our system to us about CPU, about memory. So our regulators can be modified on the fly based on that feedback. That's pretty cool. So we can also get feedback from actual multiple nodes. We have, let's say, a cluster of distributed nodes. We actually can get feedback from multiple nodes. And this feedback will be applied to the predefined regulator limits. And also the load factors from remote nodes can be either averaged or you can take the maximum number. There is a way to do it in jobs framework. I'm not going to actually go into depth on every single aspect of it. So this is how we set it up. We create, we actually start up the sampler framework by setting samplers in the configuration. Then we assign our counter regulator with a modifier in the CPU modifier that we actually have started with the sampler framework here. Sampler framework comes with built in samplers. Job sampler CPU is a built in. You don't have to worry about it. But there is a little problem with it if you're not running on Linux or Unix system. It's using built in or relying application OSmon CPU SAP, which is a CPU SAP is a lower than CPU generation supervisor process. And the problem with it is it only runs on Unix or Linux systems. So it's actually not going to work for my, it's actually not going to work for on my Mac. So what can we do? We can write our own sampler plug-ins. So job framework provides this sampler behavior and you can write your own sampler. You can be creative about what it's doing. So I was very creative and came up with the idea how I'm going to create a sampler for that will work, a CPU sampler that will work on my Mac. So what I'm going to do, I'm going to take a temperature sensor and I will attach it to my laptop. I'll definitely have to turn off the fans to make sure that the temperature reading is correct. And you can use this kind of sampler. You can actually use this kind of sampler on any system. Regardless of the system, it will just work, right? Anyone's going to try that? It's kind of cool, right? Actually, I hope not. So, but let's say we decided to do something like this and this is how we are going to implement our own sampler. We'll call it myThermosampler and it will have to implement the job sampler behavior and those four methods. And then we'll be using myThermosampler for the CPU sampler rather than the built-in one. And other what if. What if we get very different, what if we get post requests and they bring in files of very different, like of very different sizes? All of a sudden, our setting the maxQ size to a specific number is not going to really help us because if there are such hit, like if the request are of such different sizes, it's just one big request might totally take up a lot of memory. So, probably would be kind of nice to maybe create a different, like a custom Q that will be measuring the Q size based on its invites rather than number of entries. So, we'll create our memory friendly Q. It will be measured, like I said. So, and let's say we are, in our particular business case, we can't really drop the requests. We really have to serve every request that comes in because every client is important. So, we can also add another nice thing to it and let's say when the Q gets too large, we actually will start saving the requests off the disk. So, it will take longer time but it will, but we'll still not lose any requests. So, we'll implement our own Q, memory friendly Q and it will implement job, job's Q behavior and we need to implement the seven methods and it will work with the job's framework and this is how we set it up. We didn't see this parameter before mod and this is the module that is the Q implementation module and we'll put our memory friendly Q in there. So, now just think about it if you're not a functional programmer or if you're coming from Java or C sharp, just how easy would this be to do? We have, we are setting up this library and we are just telling it to use our own implementation of the Q and this is basically all you have to do. You implement it and then you just assign it. There is no class loading nightmares, no dependency injection nightmares. It's just, it's that simple. If you give it the right name, if the name matches the module name and you have that module in there, it will just work. It will be no nightmares. So, next cool thing would be to actually have our memory friendly Q react to the actual memory situation in our system. So, we could maybe design, create our own sampler called free RAM sampler and to implement it, we'll use OS-man-mem sub process. Also, all of that is built in was, Erlang comes with a lot of great tools. So, if you want to measure memory, Erlang has something like that, something like that for you. So, let's implement that free RAM sampler. So, we're going to go a little further into the implementation now. So, it's going to, like we said before, implement job sampler behavior. It will have these four methods and the sample method will actually call mem sub get system memory data and this system memory data will be assigned into this structure above it. This is exactly what get system memory data returns. So, in order to capture what we need, all we need is really free mem, that green variable there. So, to do this, to actually assign the result into the proper data structure, we basically just use Erlang pattern matching. It's super easy and it's super convenient. It would have been much more difficult. Without pattern matching, it would be much more painful to do. So, it would be also, so like, so now we need to adjust the max queue size. To do it in jobs, within the jobs framework, we can modify queue size on the fly and that's how we do it. It would be kind of nice if you didn't have to, if it was, if it could do it actually dynamically based on the sampler feedback. But in order to do this in jobs framework, it's a little trickier because it doesn't actually apply, it doesn't apply sampler results to the queues, it only applies it to the regulators. So, that's something that maybe could have been improved and I'm actually kind of looking into changing, making this change. I'm not there yet. But there is a back door and there is a way to do it. So, what you can do is just start a process and subscribe, when you start it, subscribe to the job sampler. So, now your process will be getting the sampler feedback in its mailbox. And usually it comes in at once a second. So, once your process gets the feedback, it can analyze it and it can update the memory friendly queue that we created. It's max size based on the received sampler feedback. So, that's basically the approach. And now that brings us to the end of my presentation and I would like to just have a little true or false quiz. So, it actually will cover, it will cover all of my presentation. Hopefully you remember what we were talking about it. So, question, spawning processes in an airline is very cheap. You can spawn all you want. Is it true or false? True. Now, you don't really need to worry about what these processes are doing. The airline VM just will take care of this for you and it's just going to work. Is it true or false? Yes, false because you really have to worry about how the resources on your system will react to what the process you're actually doing. So, that's something you have to worry about. Next question, load regulation is pretty trivial. You can always just, if you don't trust this frameworks and you can always, what kind of bugs they have, you should always just write your own load regulation framework and make sure that it works properly. Thank you, that's false. We saw how at first it seemed very simple. Just to counter regulation big deal, it's really easy to implement but then you start thinking, hey, there's more considerations and we went through quite a few of them and you probably have way more than I do. And then the last question, we should write CPU intensive computations and see as single units of work and let airline manage the current execution. True or false? Well, everybody got to try it. Thank you very much for listening and now it's time for questions. If you have any? No questions? All right. Please don't forget to vote and don't forget to towel. Thank you very much for listening. I'm Irina Guberman. Thank you.
 Hello, everybody. My name is Irina. I hope you're ready to get started. I'll be talking  about maximizing throughput on multi-core systems with Erlang. I just wanted to get a  feel for how many people here know a little about Erlang. And how many people are seriously  considering learning Erlang. All right. So, yes, then, let's get started. So, what we're  going to, what I'm going to talk about is an actual project I was working on at Ubiquiti  Networks. That's my previous company. And we were working on a Rust-WHTP server handling  audio configurations. So, we're going to go on a deep dive on what we had to deal with  and how we solved the problems that we run into with Erlang. So, what we have is audio  configurations coming from various audio devices or a group of devices or a whole private branch  exchange. And that's how the audio configuration looks very simple. It has a text audio greeting  and then the audio file output formats. And that's what our request will be. And we get  multiple such requests. So, when our server gets this request, we're going to be handling  the usual Rustful operations. These are not something I want to talk about. They're not  that interesting, but we're going to be talking about the fun part, the actual audio conversions.  So, the text that we get will be converted to an audio file. And then resulting audio  file will be converted to other multiple formats that we have in the request. And then once  we get the audio converted, we need to save it to Amazon Simple Storage Service, also  known as S3. And as the audios are saved to S3, we're going to save the URL in our database  so that the clients know that they are ready to be retrieved. So, that's pretty much the  flow of our request handling, that's our request handling flow. So, a couple of things we needed  to worry about is that audio conversions are very CPU intensive. And saves to S3, on the  other hand, are easy on the CPU, but might take an unpredictably long time. So, obviously,  with this kind of setup, we can have our clients waiting to get their request back immediately.  So, what we did was validate the request and send a reply back, synchronously send a reply  back to the client saying that the request is processing and then process the rest of  it asynchronously. So, let's take a look at our approach in airline. So, for every request  that came in, we created a separate financial state machine, a separate process, a separate  financial state machine. So, let's say we have audio ID 101 coming in with this request  and we spawn a financial state machine to process that one request. And we are going  to give this financial state machine ID equal to the audio ID and we'll see in just a second  why we do that. So, we have another request, so the financial state machine is processing  our audio request. Now, another audio request comes in, we spawn another financial state  machine. So, this is ID 100 and this is FSM with ID 100 as well. Now, audio ID 104 comes  in, we spawn FSM with ID 104 and it's running. And finally, another audio ID 104 request  comes in and that's bad because, I mean not bad, but it would have been bad, but because  all sorts of race conditions and problems could arise from that if you're processing  the same audio at the same time, but we know we are running this, the FSM was ID 104 is  already running, so this request will just be denied and it won't pass the validation.  So, that's why we actually keep tracks of the IDs of the FSM we are running to make sure  that we don't allow any simultaneous requests for the same audio. So, we're going to have  more of this spawn on the screen and I know it's kind of tiring, I'm bringing up these  FSMs on the screen, but just imagine how tired our server is, it just keeps getting those  requests and spawning them and it's going to be hundreds and thousands of them. 101  just finished, great. Oh, and we got another 101 request coming in. And now it's okay with  us spawning another 101 of FSM because the other one finished, so it's totally okay to  process another update. That's a lot of processes we just started, but that's, like I said,  this is just a small example, we are going to have hundreds and thousands, whatever  number of HTTP requests are coming in, for every single one of them, whatever the number  that number is, we're going to spawn a new process to just dedicate, dedicated process  to that one request. And that might sound scary to people who are not coming from airline  world, but it's perfectly okay for an airline developer to spawn as many processes as they  want because airline processes are very cheap and they're actually very fast to start. So,  you can spawn hundreds and thousands of them, it's perfectly fine. It's the airline VM that's  going to take care of properly allocating resources to the underlying OS strats. And you  as an airline developer, you don't have to worry about concurrency and multithreading  at all. You really just have to worry about your business case and if you need to spawn  a process, just go for it, it's perfectly fine. A couple of other things that I wanted  to show you how, a couple of other considerations was implementing things that I showed you  on a previous screen. So, we had our FSMs as we were spawning them, we were assigning  them an ID. So, when you spawn an airline process, it will have a process ID associated  with it, but that doesn't help us, it's just some random numbers. We wanted to have an  alias that same idea as the audio ID in our particular case. So, what we're going to do  there is a concept known as process registry in our wing and there are some built-in functions  to register a process. So, if you want to locally register a process within Pacific alias, you  can call register name and process ID. Or if you want to register your process globally,  so it can be discoverable across multiple nodes, you can call, there is a module global,  it comes with airline distribution and you can call global register name. There is a  little bit of a problem for us, was this to built-in functions. They only accept atoms  as process names and atoms are just constants with a name. So, the problem with atoms is  that atom table is never garbage collected. Then just think about it, why would you garbage  collect constants? So, in our particular case, these methods are not going to work because  we will have to convert those audio IDs to an atom and it's actually possible to do this  in our line, but it's really not advisable. You don't want to dynamically generate atoms  in our line because you're going to eventually have a serious memory leak in your node. So,  let's talk about how stable, how reliable airline is and how airline nodes are running  for years. It's very true, but if you do silly things like that, you're just going  to run out of memory. So, it's a huge node generating atoms dynamically in airline. So,  that won't work for us. This is a warning. If we need to do process registry and we have  actually created processes dynamically, we got to use something else. Luckily, there  are other libraries in airline and they're quite mature that we can use. One of them  is Gprog written by Ulf Wigger. His name we're going to hear one more time today just  a little later. It's a great library, but one important thing about it is that we can  use any term as a process alias. It actually has seven other bullet points of goodness,  but we don't care about them at the moment. The actual library we use is called CPG, Clarity  Process Groups. If you look at the read me, you will see the author says that it's very  different in the implementation comparing to Gprog, but bottom line is you can use any  term as a process alias. So, that's all we care about. It actually has some other nice  things about it. You can read the read me if you care about the specific libraries. Of  course, write your own library is also an option. So, if you do that, there is only one thing  you need to remember. You should be able to use any term as a process alias or at least  a non-atom. I'm going to go over a little bit, dive a little more into that problem.  You're not going to be always calling this method explicitly. If you use generic OTP  behaviors, OTP, I don't know if people know, I'll just briefly say that underlying OTP  is a framework that takes care of all the generic behaviors so you don't have to reinvent  the wheel. If you want to start a server, you just use gen server and implement all  the callbacks and you don't have to worry about the boilerplate code. So, in our case,  it's actually gen FSM. It's also one of the constructs that comes with modules that comes  with underlying OTP. So, one of the callbacks is called start link. That's being called  when you actually start your server. If you start with a server name, the server name  can be of three different types, local name. If you give the server name local name, the  local register method we just saw will be called implicitly. So, you're not going to  call it, but it's going to be called for you under the covers. Then, if you give it a global  name, then the global register name that we just saw will be called under the covers.  So that's bad because we're going to end up with, if we just put our audio ID, which  is an integer, or some kind of string into this method, it's just going to crash with  bad argument. And so, this won't work for us. That's just a warning that we shouldn't  be using this. But there is a third option. That one actually could work for us if we  use the proper module. One of those modules that we saw that actually use, that can use  any term or at least an atom term to register a process. So, we can, if we implement our  own module to do process registry, we need to implement these four methods and kind of  similar to how a global module does it. Next interesting thing to take a look at is  supervisors, Erlang supervisor strategies. One of, this is also a very important concept  in Erlang, supervision tree. That's what makes Erlang so robust. If a process goes down,  we always create, of course, it's up to us as Erlang developers to create a supervision  tree and make sure that the processes that are spawned are supervised. But that's pretty  much what Erlang developers always do. They have everything, every process you create  will be supervised and you kind of design your supervisor strategies. Then we are going  to look at some supervisor strategies that Erlang provides. So, first one is one for one.  The green thing on top is the supervisor. The yellow things are the running processes.  So, we have, if one, so one for one, one for one process crashes, that one process is being  restarted. The next supervisor strategy is one for all. If one process is crashed, all  of the children of that supervisor are being restarted. You could think of scenarios where  you can use, where this can be used, possibly that all of them are somewhat interrelated.  So you need to restart all of them. The next one is rest for one. It's one process crashes.  It gets restarted, but then all the children also depend on that child will be restarted  as well. So it's a little more complicated. These are pictures. These are not my drawings.  These are drawings by Fred Hubbard. He is the author of a really awesome book, Learn  You Some Erlang. If you are interested in learning Erlang, this is a perfect book to  read. It's very funny and it goes into dupes and it's going to alert you about many different  things you should know before you start your production, like before you start your production,  writing your production code. So there is a fourth type of a supervisor. It doesn't  have a picture. It's very similar to one for one. It's called simple one for one. It  is not simple at all. It's actually very, very different from the other three. The other  three supervisors are, let's call them static supervisors. When a supervisor is created,  its children are also created and it knows what it's supervising. This dynamic, simple  one for one supervisor is very different. It's kind of like if you have a startup company  and you know you need to hire many developers, but first thing you do is hire a supervisor  and then you tell him, hey, you don't have anybody to supervise right now because we  are still hiring them, but you're going to be supervising our developers. So that's  how simple one for one supervisor works. So we're going to create it. We're going to  tell it, you're going to be supervising this audio handlers, but it's not going to have  any processes to supervise at first. And I was going to, I was going to like ask you  which one of the strategies should we be using, but I already think give it out. So obviously  it's going to be simple one for one supervisor. So just in general, if you have a situation  when you are creating a, when you're creating, spawning a separate process for every request  and you don't know how many processes like that you'll be creating. Simple one for one  supervisor is probably a good idea. So next consideration is audio conversions. So what  are we going to do about that? Audio conversions are not a novel problem. Luckily we already  have, there are already C executables that they care a little bit for us. So for text  to voice conversions, we use substrel service. There is such a thing. If anyone, if you've  never done anything like that, you can use substrel. You can just play with it and have  some text converted to voice and different funny voices. And it's, you can just play  with that service. And the other service is the one that we used for converting the resulting  way file to other formats is called socks. So just basically existing executables we  use. However, if your situation is not as simple and you actually need to roll your  own, like write your own processing, the, the recommendation would be for airline developers  to write maybe a single unit of work and see rather than in the airline. And the reason  is if you need to, if you need to manipulate large data structures like huge arrays or huge  matrices and like, this is one thing you don't want to do in airline because airline data  structures are immutable. So just imagine the amount of CPU and memory that will go  into copying all this, when you make slight modifications to them. So it's better to write  those in single, like a single unit of work and see and then let airline manage the, manage  the concurrency of it. And I would like to quote my friend and big airline activist, Garrett  Smith, he's a Chicago airline meetup organizer, think of airline as a manager, not a worker.  Next problem is saves to S3. So let's say we are this busy FSM, we are dealing with  all the coordinating saves to the database and coordinating audio conversions. Now we  got, we created the S3 URL that we need and then we created, we have the audio binary  that we need to save. And, sorry. So what we are going to do, we are not going to just  sitting there waiting for, we are not just going to, we can't afford to just send this  request to S3 and sit there waiting for it. What we are going to do is spawn a child process  and this is the code to do this. This is how we are going to send the request to S3. So  we are going to spawn a child process that is going to do nothing but just sit there  waiting for that request to complete and we can just go on doing our other tasks. Important  thing to do is provide this process with our own process ID. That self method is our process  ID and we are going to send, give it to that request so it knows where to send the reply  back. So this is the process code. So it sends the S3 request, just waits for it to complete  and then when it gets the reply, it sends the result back to us. The FSM here is the process  ID is our own process. If we are the FSM, this would be our process ID. And once it's  done, it sees us to exist. So again, we solve this problem by just spawning a process. If  you are an underlying developer, you can just spawn a process if you need something to process  the synchronously. You really don't have to worry much about it. Next thing, so audio  conversions as I mentioned before, they are very CPU intensive. So what's going to happen  if we are spawning thousands of finite state machines and those finite state machines will  be spawning, like we'll be running audio conversions, thousands of them simultaneously. As powerful  as our language is, we are properly doing concurrency for us. We only have that much  CPU power. So if you don't think what can our machine do at the same time, it won't  do us any good. So what's going to happen is those audio conversions will heat up all  of the processing power and we have absolutely no room at some points in time. We'll have  absolutely no room for other important things, like say, is to the database actually accepting  HTTP requests. So, I'm going to end up in a pretty ugly situation, commonly known as  overload. So what can we do? Well, maybe an easy answer. We can just limit simultaneous  audio conversion jobs to number of CPUs minus one and reserve at least one CPU for other  high priority work. So how can we do this? We can maybe write our own code to do it.  It sounds pretty simple, right? So maybe we can write our own code to do this. But we  are pretty busy. We have a lot of business requirements to take care of. So what we did  was we used an existing library. And the library we chose, the jobs framework by Ulf  Wigger. And the rest of this talk I will actually introduce you to this library and the point  of me doing this is not necessarily for you to learn how to use this library, but more  to understand what it takes to write a library, kind of what kind of things you need to worry  about when you are writing your own library or when you are actually kind of when you  are implementing overload prevention in your code, what kind of things might come up in  the process. So what this framework is, is a job schedule for load regulation. And its  sole purpose in life is just overload prevention. So first important, first important goodie  that this framework gives us is, oh, I'm sorry, I just one thing before I go into that. This  framework was the inspiration behind this jobs framework is described fully in this  article that Ulf Wigger wrote in August of 2010. And it's actually a very interesting  and serious article on overload prevention. You might want to consider reading it. But  I'm just showing you this article just because I want to show you this diagram. This diagram  we see the failures in the US public switch telephony network out each minutes by cause.  And as obviously we see here that overload is the leading cause and failures. So I hope  this diagram is a good proof for you to consider overload prevention in your code before it  goes into production. So now let's take a look at what a job framework has to offer us.  So first, that's what we actually use the counter based regulation the way it works.  You set up how many jobs of the certain kind at the same time you want to run. And only  that many jobs will in the jobs framework will make sure only that many jobs are running.  Anything else gets queued up and only starts executing when the other jobs are done. So  I kind of have a little visual demonstration of how it works here. And what I would like  to do next is kind of deep dive into how we set it up with the jobs framework. This is  a little more, a little more airline code here, but just kind of to show you how what  it takes to set something like that was airline. So first thing we need to do is add the jobs  repository link to the job, rebar config dependencies. Rebar is the build tool in airline. And you,  the rebar config is the, it's configuration and that's how it's going to look.  Step two, we would need to set up the actual jobs configuration. So we create this audio  conversion skew for our specific case and you assign it a regulator called counter and  we give it a limit of seven. Let's say we have eight CPUs. So we decide to give it a  limit of seven. Of course, that's the number that you need to think about it depending  on what kind of hardware you'll be running your process on. And now this is the vast  complexity of our business logic. And anywhere within that vast complexity to get your audio  conversions regulated, all I had to do is just drop the call to audio conversions in  a call to the job server. And that's it. Our audio conversions are counter regulated.  Very simple. And I also am going to show you my real code. So this is my code. And right  where I needed to do the text to voice conversion, I just dropped it in the call to the job  server and it doesn't matter where you do it in your code, you wrap it in the call to  the job server. It's going to be, it's going to make sure that no more than seven jobs  like that are running. That's, I think it's pretty cool. So one last stop, we actually  have been mentioning the job server. So we need to actually have it running. So in order  to have that job server or application running, we need to add it to our application startup  list. And now you might be thinking, hey, she's, she made it sound so easy, but we actually  need to have that job server running. So I need a set extra server running in my machine  for this whole magic to work. Well, as an airline developer, you don't have to worry  about starting an extra service on your machine because really, airline is pretty, is kind  of like an, it's a pretty unique thing actually. It's like an operating system for your code.  Just think about it. So you can start any servers on your machine as you want. You don't  have to talk to your manager or your operations team manager about, oh, I need to this, nobody  has to maintain it. It's really up to you what other services you start with your node.  So we need the job server running, but that's not a problem at all. And it's actually, it  doesn't have that much overhead either. I think it's more like, it is there, but it's, it's  overhead is totally worse. The overload, it prevents you that it gives us, it's totally  worse. So the next, next great thing that job framework gives us, before we look at it,  I wanted to show you this little video. So let's pretend that these two ladies are our  HTTP server and the chocolates that are coming in our HTTP requests. So they're coming in  at a very nice steady pace and they can easily handle them. And hopefully that's how our  server is going to operate all the time. But no, look at that. That starts, this request  is starting to go faster and faster and it's not looking good anymore. Oh my gosh. And just  think about it. This could be your HTTP server in production. And this could be your HTTP  server in production in the middle of the night. And you probably don't like that. So  in order to prevent this kind of situation, the job framework gives us rate based regulation.  So again, a little visual demonstration of what rate based regulation is. The request  might come in at whatever pace and you just get, but they'll get through the server at  a very specific pace that you allow. So given a frequency F, it ensures that the rate of  accepted jobs does not exceed F. So to set up the, to set up the, so to set up our HTTP  request running at a specific pace, we're going to create a new queue called HTTP requests  and we're going to give it a rate, a regulator name rate and we give it a limit of 1000 requests  per second. Okay. That number is of course just out of the blue. You need to measure  your server performance and decide yourself what is the best rates your server can handle.  So that number, you need to really worry about it. Of course, it's much better to worry about  it before everything's in production than afterwards. Couple of, so now I showed you  two great things that the job framework brings, but I personally had some what ifs creeping  into my head. What ifs, we get a denial of service attack. Well, I guess it's okay because  you know, we have this rate regulation set up with our HTTP server. So no matter how  many requests are pounding our server, they're going to still be going through our server  at a specific pace, right? Well, unfortunately no, because all the requests that didn't make  it will be actually queuing up and that accumulating in the queue and that queue eventually will  fade and blow our server's memory. So what can we do? Job framework took care of that.  We can limit our queue size and this is how we do this. We basically just give it a max  size. So I'm just trying to show you here a very nice, sort of a very nice API. It's  very easy and you can address very numerous concerns and you can just add, so you just  have to add max size in your queue and that will hopefully save you from this scenario.  What if we have requests that have a strict timeout? Why would we keep them in the queue?  So let's say I used to work in an artifact company and we had, the RTB requests were,  had various real time bidding providers would give us very strict timeouts of like no more  than 200 milliseconds. So if your request has been sitting in the queue for longer than  that, why would you keep it there? So to deal with that, we need to limit the request playtime  in the queue. There is no point of keeping them around if they're no longer relevant.  So job framework has answered to that. Great. We can set the max time of a request in the  queue. We can give it, let's say 200 milliseconds. 200 milliseconds is the default unit. So we'll  create a new queue called RTB requests and we're going to give it next timeout of 200.  What if we also have some long running requests like payment processing requests? Those, we  don't want a timeout. We can let them run for five minutes or 10 minutes. We want to  make sure they come to, they actually complete. But even, even such requests will have a reasonable  timeout because if they're running, if they've been running for 10 minutes, probably something's  wrong. So no matter how important and long running your request might be, it's, it's  still, it still might have a reasonable timeout. So let's create a queue for those types of  requests. Let's call them payment requests queue and we'll give it a max time of 10 minutes.  So now we have two types, we actually ended up creating two queues to be able to differentiate  them by the max timeout. And what if we still want our requests, HTTP requests coming in  at 1000 a second, regardless of what kind of, whether it's a RTB request or a payment request.  By the way, these are very extreme cases. There is no way you're going to have a server  that processes this kind of requests. You probably have two different servers doing  it, but this is just an example. I made, I took some extreme cases, just an example,  not real, not a real life example. So for that scenario, the job framework has yet another  type of regulation. It's called group rate regulation. Again, this is a visual example  of what it does. It will, I don't know if it helps, but I tried. So the group rate regulator  allows us to set a cap on how many jobs from a group of queues can be executed per second.  So this is, the setup gets a little more complicated now. We create a group rate regulator with  a limit of 1000. We'll call it HTTP request rate. And now we assign this group rate regulator  to our RTB request. And then we assign this same group rate regulator to our payment request.  So now all our requests, regardless of their type, will be paced at 1000 per second. Now  audio conversions, we haven't talked about those in a while. What if suddenly something  else is happening in our machine and our CPUs are busier than normal and the CPU minus one,  which we thought would work for us, is no longer working? The answer to that is not as  trivial as before, but the jobs framework has an answer to that nonetheless. It comes  with sampler framework. And sampler framework is, what it does, it actually sends feedback  about various aspects of our system to us about CPU, about memory. So our regulators  can be modified on the fly based on that feedback. That's pretty cool. So we can also get feedback  from actual multiple nodes. We have, let's say, a cluster of distributed nodes. We actually  can get feedback from multiple nodes. And this feedback will be applied to the predefined  regulator limits. And also the load factors from remote nodes can be either averaged or  you can take the maximum number. There is a way to do it in jobs framework. I'm not  going to actually go into depth on every single aspect of it.  So this is how we set it up. We create, we actually start up the sampler framework by  setting samplers in the configuration. Then we assign our counter regulator with a modifier  in the CPU modifier that we actually have started with the sampler framework here. Sampler  framework comes with built in samplers. Job sampler CPU is a built in. You don't have  to worry about it. But there is a little problem with it if you're not running on Linux or  Unix system. It's using built in or relying application OSmon CPU SAP, which is a CPU SAP  is a lower than CPU generation supervisor process. And the problem with it is it only  runs on Unix or Linux systems. So it's actually not going to work for my, it's actually not  going to work for on my Mac. So what can we do? We can write our own sampler plug-ins.  So job framework provides this sampler behavior and you can write your own sampler. You can  be creative about what it's doing. So I was very creative and came up with the idea how  I'm going to create a sampler for that will work, a CPU sampler that will work on my Mac.  So what I'm going to do, I'm going to take a temperature sensor and I will attach it  to my laptop. I'll definitely have to turn off the fans to make sure that the temperature  reading is correct. And you can use this kind of sampler. You can actually use this kind  of sampler on any system. Regardless of the system, it will just work, right? Anyone's  going to try that? It's kind of cool, right? Actually, I hope not. So, but let's say we  decided to do something like this and this is how we are going to implement our own sampler.  We'll call it myThermosampler and it will have to implement the job sampler behavior  and those four methods. And then we'll be using myThermosampler for the CPU sampler  rather than the built-in one. And other what if. What if we get very different, what if  we get post requests and they bring in files of very different, like of very different  sizes? All of a sudden, our setting the maxQ size to a specific number is not going to  really help us because if there are such hit, like if the request are of such different  sizes, it's just one big request might totally take up a lot of memory. So, probably would  be kind of nice to maybe create a different, like a custom Q that will be measuring the  Q size based on its invites rather than number of entries. So, we'll create our memory friendly  Q. It will be measured, like I said. So, and let's say we are, in our particular business  case, we can't really drop the requests. We really have to serve every request that comes  in because every client is important. So, we can also add another nice thing to it and  let's say when the Q gets too large, we actually will start saving the requests off the disk.  So, it will take longer time but it will, but we'll still not lose any requests. So,  we'll implement our own Q, memory friendly Q and it will implement job, job's Q behavior  and we need to implement the seven methods and it will work with the job's framework  and this is how we set it up. We didn't see this parameter before mod and this is the  module that is the Q implementation module and we'll put our memory friendly Q in there.  So, now just think about it if you're not a functional programmer or if you're coming  from Java or C sharp, just how easy would this be to do? We have, we are setting up  this library and we are just telling it to use our own implementation of the Q and this  is basically all you have to do. You implement it and then you just assign it. There is no  class loading nightmares, no dependency injection nightmares. It's just, it's that simple. If  you give it the right name, if the name matches the module name and you have that module in  there, it will just work. It will be no nightmares. So, next cool thing would be to actually have  our memory friendly Q react to the actual memory situation in our system. So, we could  maybe design, create our own sampler called free RAM sampler and to implement it, we'll  use OS-man-mem sub process. Also, all of that is built in was, Erlang comes with a lot of  great tools. So, if you want to measure memory, Erlang has something like that, something  like that for you. So, let's implement that free RAM sampler. So, we're going to go a  little further into the implementation now. So, it's going to, like we said before, implement  job sampler behavior. It will have these four methods and the sample method will actually  call mem sub get system memory data and this system memory data will be assigned into this  structure above it. This is exactly what get system memory data returns. So, in order to  capture what we need, all we need is really free mem, that green variable there. So, to  do this, to actually assign the result into the proper data structure, we basically just  use Erlang pattern matching. It's super easy and it's super convenient. It would have been  much more difficult. Without pattern matching, it would be much more painful to do. So, it  would be also, so like, so now we need to adjust the max queue size. To do it in jobs,  within the jobs framework, we can modify queue size on the fly and that's how we do it. It  would be kind of nice if you didn't have to, if it was, if it could do it actually dynamically  based on the sampler feedback. But in order to do this in jobs framework, it's a little  trickier because it doesn't actually apply, it doesn't apply sampler results to the queues,  it only applies it to the regulators. So, that's something that maybe could have been improved  and I'm actually kind of looking into changing, making this change. I'm not there yet. But  there is a back door and there is a way to do it. So, what you can do is just start a  process and subscribe, when you start it, subscribe to the job sampler. So, now your  process will be getting the sampler feedback in its mailbox. And usually it comes in at  once a second. So, once your process gets the feedback, it can analyze it and it can  update the memory friendly queue that we created. It's max size based on the received sampler  feedback. So, that's basically the approach. And now that brings us to the end of my presentation  and I would like to just have a little true or false quiz. So, it actually will cover,  it will cover all of my presentation. Hopefully you remember what we were talking about it.  So, question, spawning processes in an airline is very cheap. You can spawn all you want.  Is it true or false? True. Now, you don't really need to worry about what these processes  are doing. The airline VM just will take care of this for you and it's just going to work.  Is it true or false? Yes, false because you really have to worry about how the resources  on your system will react to what the process you're actually doing. So, that's something  you have to worry about. Next question, load regulation is pretty trivial. You can always  just, if you don't trust this frameworks and you can always, what kind of bugs they have,  you should always just write your own load regulation framework and make sure that it  works properly. Thank you, that's false. We saw how at first it seemed very simple. Just  to counter regulation big deal, it's really easy to implement but then you start thinking,  hey, there's more considerations and we went through quite a few of them and you probably  have way more than I do. And then the last question, we should write CPU intensive computations  and see as single units of work and let airline manage the current execution. True or false?  Well, everybody got to try it. Thank you very much for listening and now it's time for questions.  If you have any? No questions? All right. Please don't forget to vote and don't forget  to towel. Thank you very much for listening. I'm Irina Guberman.  Thank you.
Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_0_875.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_876_2783.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_2784_3839.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_3840_4859.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_4860_5243.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_5244_5747.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_5748_6347.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_6348_6875.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_6876_7355.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_7356_7403.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_7404_7475.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_7476_7583.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_7584_7739.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_7740_8015.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_8016_15443.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_15444_18443.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_18444_19835.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_19836_20363.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_20364_23243.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_23244_26087.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_26088_27035.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_27036_27227.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_27228_27407.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_27408_27875.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_27876_28967.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_28968_29339.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_29340_30923.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_30924_32099.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_32100_32987.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_32988_33323.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_33324_34043.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_34044_34595.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_34596_36215.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_36216_36863.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_36864_39767.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_39768_40199.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_40200_40859.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_40860_41171.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_41172_41987.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_41988_43223.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_43224_43391.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_43392_44279.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_44280_44831.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_44832_48035.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_48036_48647.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_48648_49319.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_49320_49955.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_49956_51179.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_51180_51983.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_51984_52799.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_52800_53183.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_53184_54107.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_54108_54431.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_54432_54599.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_54600_54647.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_54648_56507.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_56508_57983.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_57984_58835.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_58836_60311.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_60312_65015.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_65016_65339.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_65340_65699.png Maximizing_throughput_on_Multicore_Systems_-_Irina_Guberman_65700_66762.png