Inside ActiveJob
CC Attribution - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
D'Antonio, Jerry
ActiveJob made a huge impact when it landed Rails 4.2. Most job processors support it and many developers use it. But few ever need to dig into the internals. How exactly does ActiveJob allow us to execute performant, thread-safe, asynchronous jobs in a language not known for concurrency? This talk will answer that question. We'll build our own asynchronous job processor from scratch and along the way we'll take a deep dive into queues, job serialization, scheduled tasks, and Ruby's memory model.
 So it is time to get started. So this is Inside Active Job. This is the Beyond the Magic track. My name is Jerry DeAntonio. I live and work in Akron, Ohio. If you are an NBA fan, you have probably heard of Akron. There is a local kid who has done pretty well from the NBA. I went to school about 10 minutes from where I live. I work at Test Double. You may have heard of Test Double. Justin Searles, one of our founders. It was on the program committee for Rails Coffee. Test Double, our mission is to improve the way the world builds software. I know that sounds very audacious, but we truly believe that every programmer has it in themselves to do that. I believe every person here has it in themselves to do that, and that is why you are here. Definitely it has been a great company to work for, and I am very proud to represent Test Double here. Personally, one thing I have done, my biggest claim to fame lately is I created a Ruby gem called Concurrent Ruby. You may have heard of Concurrent Ruby because it is used in some very well-known projects like Rails. Concurrent Ruby is a dependency of Action Cable. In Rails 4 and 5, it is used by Sprockets. I also use some gems like Sidekick, Sucker Punch, elastic search and Log Stash Utilities. Microsoft is your Ruby tool. Much of what I am going to be talking about today draws my experience from that, but this is not going to be a sales pitch for that. This is going to be about ActiveJob and Rails itself. Because this is beyond the magic track, this is not going to be an introductory topic. This is going to be a deep dive into the internals of ActiveJob. I have had to make a couple of assumptions in doing this. I am assuming that if you are here, you have used ActiveJob, probably in production, you have used one of the supported job processors, you have some understanding of concurrency and parallelism. If you need a better introduction to ActiveJob itself, I highly recommend the Rails guides. The Rails guides are excellent at this and provide a lot of great information. If you need an introduction to concurrency within Ruby itself, same-less plug. I did give a presentation last fall at RubyConf called Everything You Know About the GIL is Wrong. That video is available on YouTube and that could be an introduction into that. So with that, let's jump into what is ActiveJob. So in order to get into the internals of this, I need to briefly remind us of what it is and where it came from. So ActiveJob, according to Rails guides, the definition is this. ActiveJob is a framework for declaring jobs and making them run on a variety of queuing back ends. So ActiveJob can be everything from regularly scheduled cleanups to billing charges, mailings, anything that can be chopped up in small units of work and run in parallel. The couple key terms there. It's a framework. We're going to talk more about this, but asynchronous job processing, pre-existed ActiveJob. There were things like Backburner Delay Job, Q Rescue Sidekick, Sneaker, Sucker Punch. Many of these things existed before ActiveJob was created and ActiveJob came along as a way of unifying those. ActiveJob helps us schedule tasks to be run later. That was mentioned briefly this morning in the keynote that when you don't want to block the currently running PREB request and you want something to happen later, you use ActiveJob in order to make that happen. And that can happen through what we call ASAP processing, which is where you get to this as soon as you can or by scheduling it at a later date and time, potentially. And this is also good to support full parallelism. Some of the job processors are multi-threaded. Many of them, however, actually are forked. I'll talk about more and can run multiple processes on machine and scale across multiple processors. And in some cases across multiple machines. So the impetus for ActiveJob is that background job processors exist to solve a problem. We have these long-running tasks that we don't want to block the web request. So we want to be able to send a response back to the user and get the page rendered for them. And some of these tasks then occur after that. So for example, if I'm sending an email, this email takes time. It's a sin to begin with. Why should I block the web request to make sure that email posts when I can send the response back and have that post shortly thereafter? So ActiveJob supports that. And the processors behind that support that. So like I said, ActiveJob, this is going to be important when we get into the internals, ActiveJob came later. And there are all of these job processors. Each one was unique. And each one, they all did virtually the same thing. They had slightly different capabilities. And when it got to a different late, they all saw the same problem. Right. So ActiveJob was created to provide a common abstraction layer over those processors that allow the Rails developer to not worry about the specific implementation. Right. This sounds familiar. This is not dissimilar to what ActiveRecord does. All right. Relational databases existed. ActiveRecord created an abstraction layer over that that allows us to run, use different databases to fairly switch between different databases if necessary. Most importantly, run different databases and test, prod, and dev. Right. ActiveJob does the same thing. It allows the abstraction layer that will allow us to choose different processors, change different processors as our needs occur, and run different processors and test, development, and production. Right. And so ActiveJob had to do that while supporting the existing tools that people were already using. So, according to RailsGuides, is picking your key back end becomes more of an operational concern. Use the developer, don't care which back end you use is being used. You simply write the jobs and let the application and then use whichever back end in whatever environment makes the most sense. So, because we're looking at some code, I want to real briefly remind us what the code looks like for ActiveJob before we jump into the internal. So, this is a simple job class. This should look familiar to everybody. The important things are that this class extends ActiveJob base and that it has a method called perform. Right. Most of what ActiveJob does is encapsulate it in the ActiveJob base class, which goes and will eventually, as we look through the details, call this perform method on your, on object of this class when the job actually runs. And look at those details. Right. And as a reminder, the way we configure our back end is we use this JobQ, ActiveJobQAdapter configuration option within our application RB. Now, InsideJob is what I'm going to call the adapter we're going to build here. We're actually going to build in here a real adapter that is functional. Right. So, the, all of the adapters that are supported by Rails have a symbol that follows normal Rails inflections that maps the adapter name to what you set the convict value for. So, if InsideJob existed as a supported adapter in Rails, this would be how you would set that. Right. Then, that's how you configure which back end you want to use. And then later when you want to actually do something later, you call the perform later method on your class, passing it one or more parameters. Okay. And that should look familiar to everybody. And if you want to schedule the job for a certain time, then you can use the set function to specify when and there's a number of different ways you can do that. So, that's just a reminder of what we see on the front of ActiveJob. All of that should look familiar to everybody. We're going to talk about is what goes on behind that when you make this perform later call. Okay. So, I guess we're going to build a asynchronous back end here, right up here, during this presentation. One that actually works and is functional and will meet a, it's minimal, but it will meet the requirements of ActiveJob and show us how this works. So, a couple of things just to give a sense of where we're coming from. Like I mentioned, there are multi-threaded adapters and there are forked adapters. Multi-threaded adapters run your job in the same process as the Rails app itself. Okay. There are a couple that do that. The advantage of that is those can be very fast and you don't have to spawn separate processes and manage separate processes. All right. We all know that MRI Ruby does have some constraints with concern to concurrency, but it's not as bad as most people think. That's what I talked about at RailsConf last fall. And since most, MRI Ruby is very good at multi-threaded operations when you're doing blocking I.O. And most of the tasks we're going to be making these background jobs for are doing blocking I.O. They're sending emails. They're posting things to other APIs. And so, since they tend to do blocking I.O., they tend to work very well with Ruby's concurrency model. So, a threaded back end is simpler because we don't have to manage separate processes. Many of them, however, do use or they do spawn fork separate processes where you have to run separate worker processes. Those give you full parallelism, but they require active management of those processes. So, for what we're going to build here, we're just going to do a multi-threaded one because I can do that very easily. And it will demonstrate all the things we're going to do. And we're going to use thread pools for that. Most job processors will also persist the job data into some sort of data store. Redis is very popular for this. The reason for doing that is that if your Rails process exits, either on purpose or by crashing, if all of your job data is in memory, you're going to lose it and those jobs will never run. So generally speaking, for production, you want to have a job processor that does store the job data in some sort of external data store to allow it to persist beyond restarts. We're not going to do that here mainly because in simplicity, I want to demonstrate what goes on in active job. We don't have to go to that level of effort. So, our job processor will not persist through a data store. So it makes it good for testing and development. But not necessarily we wouldn't use what I'm going to build here today for production. So in order to do this, we're going to need three pieces. The first one is active job core. This is provided by active job itself and it is the job metadata. I want to talk about this more, but it is the thing that defines the job that you need to perform later on. It is probably the, I'd say the most important piece of all of this because it's the glue that binds everything else together. The two pieces we're going to build today are the QAdapter and the Job Runner. Remember, active job came about after the Job Runners. So the Job Runner is independent and it provides the asynchronous behavior. The Job Runner actually exists as a separate thing. Sidekick is a separate thing. Sucker Punch is a separate thing. You install those separately. The QAdapter has the, its only responsibility is to marshal the job data into the asynchronous job processor. So the job processor provides the asynchronous behavior and the QAdapter marshals between your Rails app and that job processor. And those are the two pieces we're going to build here today, the QAdapter and the Job Runner. For all of the Job Runners supported by Rails, the QAdapter is actually in the Rails code base. Okay. If you go into GitHub, go into the Rails code base and look at active job, you'll see that there's a folder of QAdapters and there's one QAdapter in there for each of the processors that Rails supports. There's also a set of unit tests as part of the Rails code base that will run against every one of these job processors on every commit. And they ensure that all of the supported job processors meet the minimum requirements of active job. The one we're going to build today actually will pass that test suite. And once it's in the job, it will pass that test suite. So strictly speaking, the Rails core team has responsibility for the QAdapters and for that test suite. But knowing from experience, the people who create the Job Runners themselves work very closely with Rails to make sure that those adapters are up to date and work well with the processors. So let's stop then and talk about the active job core class. Like you said, this is the glue that ties it all together. It's not obvious. So this is the job metadata. It is an object that represents all of the information about the job you've posed. It carries with it the proc that needs to be run. It carries with it things like the Q and the priority was stopped in a minute. And it carries with it all of that metadata. It provides two very important methods which I'll talk more in a minute. But they are the serialized and deserialized methods. These are very, very critical. I'll talk about them in a minute. The job metadata itself, there are several attributes on this object which we will look at and use internally within active job. These are not things that you as a Rails developer have to know about, but these are things that when turned inside of active job are very important. One of them is the Q name. Most of us should be familiar with that. You can specify when you create a postman these jobs what Q should run against. And if you don't specify, it's the default Q. Priority, some job processes support prioritization where higher-party jobs run first. We're not going to support prioritization in ours. That's optional. But the priority would be attached to this as well. If you schedule a job to run at a specific time, you get an attribute called schedule that which tells you when. We'll look at that because we are going to do scheduled jobs. The job ID is internal to Rails. And there's a unique ID within the Rails instance itself that identifies each job. Rails uses that within active job to track each one of these. The provider job ID is one that you can provide within your job processor. So if we want to do it within our job process, if we want to have our own kind of ID system that made sense for us and worked, we can then attach it to the job metadata under the provider job ID. So Rails does not create that. We would create that ourselves. We're not going to use the provider job ID today because it's not essential, but it is available and it's something we would add. So let's actually build a QAdapter. We're going to go outside in. So like I said, the QAdapter is responsible for marshaling data into the job processor. The job processor is the more interesting piece. We'll look at that in a minute. But we're going to start with the QAdapter. And we're sort of going to pseudo-TDD this. The QAdapter, most of the QAdapters were written when active job was created because the job processors already existed. And they had to handle that marshaling. In our case, because we don't have a QAdapter, or excuse me, we don't have a processor yet, we can decide what the API is going to look like. So within our QAdapter, we only need two methods. It's very simple. One is NQ and the other is NQAt. The NQ method takes that job object we looked at a minute ago, and it marshals that into our processor. And the NQAt takes the job and a timestamp and marshals that into our job processor. So notice in this case, I've decided to make the API very simple. We're going to create a thing called inside job. We're going to have class methods NQ and NQAt. We're going to pass the serialized job. We're going to pass the QName. And in the case of the NQAt, we're going to pass the timestamp. So a couple of things to note. One, this is not very OO. These are class level methods that we're calling on this class. And I did that because I want to emphasize the stateless nature of this. This is very critical to understand. Active job is by its nature stateless. The state for your job is encapsulated in that job object. All of the metadata about the job, everything related to that job, all of your state is in that that we're passing through. The actual QAdapter itself is inherently stateless. Its job is just to... In your notice, we even call a class level method when we post the job. Because we're sending this thing to happen later on. It's a fire and forget. We're not creating anything that's going to be persisted. And in fact, any kind of stateful behavior here would be potentially thread unsafe. So we're just going to call these class methods and throw this data at it. And then we'll build those class methods in a minute. And that's all it really takes to build a QAdapter. Now, one thing that's very important here is the serialized method. And I have to go into this in a little detail. The reason why we call this serialized method is twofold. First off, and less importantly, is thread safety. Remember, Ruby is a shared memory language that has object references. So if we have maintained a reference to anything that was passed into that, and we hold onto that reference when this thing finally goes and gets processed later on, if it's processed in the same process on another thread, we run it a potentially not thread safe behavior. Now, the normal usage pattern makes that not really a big deal. But if we serialize the job into a representation of that, we then let go of those references and make it thread safe. There's one important reason, though. And the most important reason is for consistency. Remember, we want to be able to work with multiple job processors in prod and dev and even test we want to be able to. So when those job processors are going to persist into a data store such as Redis, they must serialize somehow. Maybe this can't take a Ruby object and throw it into Redis. Or throw it into a traditional database, we have to serialize it somehow. That's sounding bad. So if every job processor created its own serialization method, we could potentially run into problems when we switch between these. We don't want to have hidden errors where we run this in tests and we run it in dev, and all the serialization works, and then we run it in production with a different processor and the serialization fails or does something different. So ActiveJob provides one common serialization routine method and one deserialization method so all of the job processors can choose to serialize the same way. And in so doing, that will allow make sure we reduce one potential set of errors when we move between job processors. So we are going to serialize here, even though, you know, this is the simplified version and we're not storing this in a data store. We want to do that serialization to make sure we get that consistency across processors. So internally, like we said, we need to do two things. We need to provide inside the QAdapter, I'm sorry, I moved on to the job processor now. So we have the QAdapter, now we need the job processor. The job processor's responsibility is to provide the asynchronous behavior and that asynchronous behavior is QDependent. So we want to have multiple Qs and have each Q process a different set of jobs. So for this, we're just going to use a, what we need is we need to be able to post jobs into different Qs and have them behave asynchronously. We're going to use a simple thread pool for this, right? Because within the context of this simplified application, a thread pool works fine. A thread pool has its own Q and therefore, by creating a separate thread pool for each Q, we do get a separate Q for these different jobs. We just have to map the thread pool to the Q name, which we'll see in a minute. And then, obviously, a thread pool has one or more threads and therefore provides asynchronous behavior. So we can very simply deal with these needs of the Q and asynchronous behavior by just creating a thread pool. Okay? So what we're going to do here is we're going to create the thread pool, but because this is all very multithreaded and therefore needs to be thread safe, not only are we creating these threads within our job processor itself, but because Rails can be run under multithreaded web servers, we need to go through a couple of hoops in order to get some thread safety here. So we're going to use a concurrent map class. This is similar to a Ruby hash and support similar APIs, but it has some additional behaviors. One, it's thread safe, but it also has some additional behaviors to make that work. Hopefully, most of you know that with Ruby's hash, when you create a new hash, you can pass a block to the initializer and that block will be called if the key does not exist. And that block will then be to initialize that key. So what we're going to do is whenever we try and retrieve a thread pool from our map of Qs, if it doesn't exist, we're going to create a new thread pool at that time. So we'll lazily create our thread pools as new Qs are needed. Basically do this together and just see kind of one way that you might make this pass. So this compute of absent is just a necessary thing in order to provide the atomicity and synchronicity that we need to have in order to create this new thread pool in a thread safe manner. So there's some concurrency needs there, but the end result is that's basically like creating a hash and passing a block in to the constructor. And then we're going to have a create thread pool class. In this case, we're just going to create a cache thread pool. A cache thread pool is the simplest kind of thread pool we can create rather than getting into the details of all the different configurations we could do. Basically a cache thread pool has an unlimited Q size. It will grow and add more threads as needed. And with threads become idle, it will shut them down and remove them. So over time, we'll get an optimal number of threads in that, which for our simplified processor is fine. So I mentioned that we need an NQ method inside our job processor. It's going to look like this. Basically, when we NQ this job, more jobs NQ, we're going to simply post the job to the thread pool. And when the thread pool pulls it off, we're going to call active job base execute. That's the important part right there, active job base execute. The first line, the Qs, the posts, that's just getting the thread pool, creating a new one if necessary, then posting that to be run by the thread pool whenever the thread pool has an available thread. Active job base execute is responsible for actually interrogating the job, looking up our specific class to process that job, and then posting and calling the perform method on an instance of that and passing in the arguments. So when you, in your class, early on we saw we create that perform method and it takes a set of arguments and it runs, active job base handles the interrogation of the job, creating an instance of that and calling that method. All we need to do is call the execute on that in our, when our thread pool takes this and runs it later on. And that's all it takes. Active job handles that, like I said, the internals of that. And that right there is enough for us to actually post asynchronous jobs that are performing in ASAP way in a real environment. Now for the NQ for later, it's a little more complicated. Right now. We have to get into the time. You're going to commit. So fortunately we do have at our disposal a high level abstraction that handles these kinds of scheduled tasks. And, coincidentally, it's called schedule task. All right. So the internals of schedule task are sort of beyond the scope of this, but the idea is a schedule task will take a number of seconds in the future that something is supposed to occur and it will queue it up and it will add that roughly that time. It will then pass it on to a thread pool to make it work. Where we are. So when we notice when we actually use our perform at and we use that set method, right, Rails provides a lot of convenience things for allowing us to specify when the job happens in the future. Right. Rails gives us all those great time helpers that we like, you know, one day from now and, you know, one week from now and at certain times and so forth. Active job is responsible for taking all of those convenience things that we use as Rails developers and converting them into a number of seconds in the future when this runs. Okay. So by time we get this, we already have the number of seconds in the future. So in our job process, you don't have to worry about all of those wonderful date utilities that Rails has. Rails does that for us. Right. So in this case, it's really convenient for us because schedule task, not coincidentally, takes just a list, a number of seconds in the future when the thing should run. And in this case, normally within concurrent Ruby, all of the high level abstractions run on global thread pools. So you don't have to worry about managing your thread pools. In fact, most developers should never use a thread pool directly. All right. Most libraries that provide thread pools provide them internally and provide high level abstractions that allow, that use those thread pools. So a normal circumstances, a schedule task or a future promise, any of these things, would use the global thread pool. But in this case, we need a specific thread pool because that thread pool represents our queue. Fine. I'm just going to add. So all of the high level abstractions in concurrent Ruby support the tendency injection of a thread pool. So in this case, this executor option, which is very common, is a way of saying, when you do run this thing, run it on this specific thread pool. So. So what we're doing here is saying, look, we know how many seconds in the future this thing needs to run. We know which thread pool we want it to run on. Just go and handle that. Schedule task handles that. And at the time the thing needs to run, it'll then grab that job and it will run this block. And we're going to do the same thing we did before. It's just called active job base.execute. That execute method doesn't know anything about the sinker's behavior. It just knows now is the time to execute that. Same thing we saw a minute ago. And just in case we somehow get a time value that's not in the future, we're going to just check that delay and we're going to post it directly if somebody's not in the future. And again, that's all it takes to post the task later on. Rails handles all the time sensitive stuff. We just need to make sure that we can do it at that time in the future. And believe it or not, that in its entirety is a functional asynchronous job process. So the next slide doesn't have a bunch more code on it that we normally see. And I'm putting this on one slide because I want you to see just how simple this can actually be. That in fact a real functioning asynchronous job processor can in fact fit on one slide. And this is basically it. We have a class called inside job. We have our Qs constant where we have this thread safe map where we're going to keep track of all of our thread pools. We're going to have that create thread pool method which will just return the pool we want. Then we have our NQ behavior which just throws the job onto the thread pool. And then we have NQ at which actually looks at that delay and that time stamp and gives it to a scheduled task. And that right there is actually a fully functioning asynchronous job processor that plugs in that can work with active job. And like I said, the other part was the QAdapter. And remember the QAdapter just looks like this, right? It was simply when active job calls NQ or NQ at simply post this thing off into my job processor. So that's it. So believe it or not, that is actually a fully functional asynchronous job processor that will work with active job and could be used in test or development in order to actually get asynchronous behavior without having to install registers under deep dependencies. So the next question you're probably going to ask is, all right, Jerry, are you going to put this code up online so we can look at it like this? And the answer is yes. If you want to see this code, you can find it in a very convenient place and that's Rails. The genesis of this presentation was that last fall I went to the Rails team and said, you know, it would be really useful if we had a simple asynchronous job processor in Rails 5. As you all know, we can in our config specify the inline adapter. The inline adapter will go and it will run the job synchronously so we don't have to deal with those underneath dependencies. But the problem with that is it's not real asynchronous behavior. And if we're using the inline adapter in test or development, we can sometimes mask problems by not having real asynchronous behavior. They said, why don't we just build a simple one and we'll call it async job. We'll make the, instead of inside job, it'll be async job, we'll make the symbol just async. And why don't we allow people in test and dev to run these jobs really asynchronously in order to potentially find bugs in them. And the Rails team said, that's a really good idea. And they worked with me and we got this merged into Rails 5 last fall. So if you use Rails 5 and you use the async processor, this is basically what you're going to have. This code was lifted almost line by line from the original implementation of that. Now, since then, the Rails team has done some refactoring on that. So if you go and look at the implementation now, it'll look a little bit differently. So just to give you some context of what you see different, they decided to collapse things into one file. When I originally wrote this, I had two files, one for the QAdapter and one for the job processor to sort of mirror that normal behavior you would have of the QAdapter and the processor being separate. They collapsed them into one file because they're very short and didn't need to be two. They renamed some stuff to go along with better Rails conventions. They are assigning that provider job ID, again, in this case, we're not really needing it, but having that does, again, provide for greater consistency with the production ones. And they decided to throw one thread pool for everything and it's fenced with having multiple queues. Because, again, in test, all we care that these things happen asynchronously. We don't particularly care about configuring the queues for various different behaviors. So if you go look at it now, you'll still see async job and it will do exactly what we showed before and it's right now available in Rails 5. So if I've piqued your interest and you want to learn more about this and see other things that you can do with this, the two things I would suggest you look at more deeply are Sucker Punch and Sidekick. Sucker Punch is a threaded in-memory async, just a job processor. It does a lot of what this does, but does it way better and more fully. The creator of this tells me that the main use case is if you want to send emails from a hosting provider, a one-click hosting provider like Heroku, you can fire off these emails because there's not a lot of high cost of failure if that thing goes down. And so those jobs are just retained in memory, not persisted through Datastore. But Sucker Punch does use thread pools just like this does. It does map those queue names to thread pools and provides some configuration of those. And also does some really cool things where it decorates every job with a job runner class that does certain cool things like track the number of successful jobs, track the number of failed jobs, handle errors, and do things in this nature. So it's a really good example of how you can decorate a job when you push it into a thread pool and do some really cool things. Like I said, for most of us, we shouldn't use thread pools directly. The high level abstractions in the concurrency libraries provide those capabilities. But this is a really good example of how you can do that. And also, Sucker Punch does some really cool shutdown behavior where if the Rails app is shut down for some reason, it will look at the number of the jobs that are still running and try and allow the jobs to execute completely before shutting down and some other things. So there's some cool stuff in there. Sucker Punch uses a lot of the tools that we saw in here. It uses concur Ruby thread pools. It uses concur Ruby's scheduled task. Another great one is, of course, Sidekick. Sidekick is also an in-memory, I'm assuming, it's also a multi-threaded job processor. It does not use, it does persist all your stuff to a data store so that way your job data will persist beyond a restart of the application. It does not use thread pools the way we saw here. Sidekick actually spawns its own threads and manages its own threads, but it still deals with all those same things with the internals of an active job. And of course, Sidekick has a whole bunch of additional features. Like I said, Sidekick doesn't use concur Ruby's thread pools, but it does use concur Ruby for some of the low level synchronization and atomicity stuff, thread safety stuff you saw here. So there's two great examples. If you want to look at this further, then go look at those code bases and see beyond what we've done here. So with that, I just want to say, again, I work for TestDouble. We are hiring, and we are also for hire. So if you, we love talking to people about software development and about software and about how we can all improve software. So if you'd love to chat with us, by all means, reach out to us. You can find us on email, social media. Myself and Justin will be here for the rest of the conference. Like Justin will be speaking on Thursday in the afternoon at 3.30 in, he's going to be talking about RSpec in Rails 5. So I've got stickers up here. I've got stickers in my bag. I hope you had a chance to talk with you sometime before the conference is over. And with that, again, my name is Jerry DeAntonio. Thank you for having me. So I do have five minutes if anybody has questions I see or don't want them to run out. That's cool. I'm hungry too. So the question was resource contention within the job itself. If you have multiple threads running simultaneously and trying to do things, all of the asynchronous behaviors provided by the job processor itself. So all active job does is provide the compatibility layer. It's important that the job processors themselves handle all of the concurrency, any kind of locking or synchronization that is necessary. But generally speaking, if you follow the best practices, a lot of that contention goes away. So for example, you're not passing an active record object. I did it. You're passing an ID, which you can then use to pull that up later on. We're serializing the jobs so that we're not storing references. But ultimately it is up to the job processor itself to be thread safe. So the question was, would you be able to use multiple job processors simultaneously? And the answer is yes, but not through active job. Active job only allows you to specify one handler. However, all, as far as I know, all, I'll say most, but as far as I know, it's all of the job processors can be used outside the context of active job. So for example, you might specify sidekick for being your main job processor, but say for certain things you want to use sucker punch, you would then just instant sucker punch directly. And so you can do it that way. And I don't know, I can't imagine why Rails would change that. But again, it's very possible. Yeah. So the question was, could we subclass active job and have two different runners? I guess, again, it's ruby. We could probably do anything we want. But there's that one configuration value within the application config. I guess we could specify, of course, we can create our own configuration values. We could create some new ones, grab those and do something of that nature. I'm sure it would be possible, but it's not something that would be directly or easily supported by Rails in active job itself. So multi-threading is just when you have multiple threads, right? A thread pool. So the question was, difference between multi-threading in general and a thread pool. A thread pool is a managed thing where the queue and all of the threads are managed by the object itself. So one of the things, like I said, so I could spawn my own threads just by calling thread.new, right? What happens if those crash? What happens if I want new threads? What happens if I have idle threads? How do I enqueue those things there? There's a lot of plumbing involved in that. We can always spawn multiple threads, but in order to manage that, there's a lot of extra stuff. How do we handle exceptions? If you throw an exception on a thread, it will crash the thread. How do you handle that? So a thread pool takes all of that, puts it in one object with some very well-known, very common cross-language algorithms and manages those things. So you create a thread pool, you give it a set of configuration parameters, things like how many threads to run at a minimum, how many to maximum, how many things can you enqueue, if the queue gets full, what do you do, if you can't generate it. The outbreak system won't give you more threads than what you do, and it handles all of that for you. So all you do is just create this one abstraction, the thread pool, and you throw stuff at it, and it manages all of that in queuing and dequeuing. How to find those migrations. If threads crash, it will handle that and so forth. So there is some overhead in the thread pool itself because of all of that. But just like anything else, that overhead comes at the value of making you not worry about those things. So generally speaking, you start with high-level abstractions that use the thread pools, so you don't have to worry about that. Then maybe later on, you specify your own thread pools and inject them and take a better control. And then maybe if you're this guy over there, you just write your own threading yourself. But that's sort of the progression. And that is a fantastic question. The question is, how does it handle exceptions? What we did here does not handle exceptions very well at all. The thread pool itself will protect itself from any kind of exception on the thread. So thread pool will not allow its threads to die because of exceptions. Thread pool doesn't do much with them. Again, this is one of the reasons to use that, to use a high-level abstraction. Because if you use a future or promise or active agent, those things have consistent idiomatic ways. Idiomatic ways of handling things like return values and errors and so forth. So if you look at Sucker Punch, the job decorator class in there actually handles the exception. It's like a nice little bit of Rails magic. On the thread itself before it bubbles up and then doing things with it. So again, the high-level abstractions are the ones where people provide you with better error handling and consistent idiomatic. They're dealing with return values and minimizing the weight on things and so forth. And again, that's why you should always start with high-level abstractions and then only inject the thread pool. Thread pool is meant to be the very lowest level in that and just provides sort of the engine. So like I said, the actual job processor itself will handle things like errors. So if you look at whatever you use, which one of those supported job processors you use, they are all doing the error handling in this case. Because this is the bare bones minimum, I'm not handling errors at all. Your job is just going to die, you'll not know about it. But again, this is meant to be minimal and trivial. But if you use any one of the full-blown production-ready job processors, they will handle the decoration of that job and they will handle the errors and they will have their own way of doing that. That's one thing that ActiveJob does not include is a consistent way of handling errors. So of course, you could always put your own error handling in your perform method and handle it there, but ActiveJob doesn't really do that for you. I wouldn't do one. I mean, in terms of production, the ones that are out there are very fantastic. They're very mature. They've been used a lot. The reason for this one was to put it in Rails itself so that for development and testing, you can run your tasks synchronously and get a better understanding of how they're going to work in production. One thing, I've talked to a lot of people. In fact, one of the... If you go back and look at that commit and you look at the actual discussion around that PR, it was actually DHH himself who said, hey, I really like this idea. I generally install Sucker Punch for dev and test. It would be nice if I could just do that within Rails and not have to have that extra dependency. Sucker Punch is great. I've worked with Brandon, the creator of Sucker Punch, and he's fantastic. But it's an extra dependency for just dev and test. Rails have done a great job of providing the inline adapter for dev and test. So it makes sense for Rails to provide that simple async one. And so we just minimized what we actually need and put it in Rails itself. So now for dev and test, you can do that and get a better sense of the real behavior in production, what it might be. And how about the... I'm sorry? It's in Rails 5 now. So all you need to do is in Rails 5 is just say async in your config and it's there. Anything else? All right, thank you very much.
 So it is time to get started. So this is Inside Active Job. This is the Beyond the Magic track. My name is Jerry DeAntonio.  I live and work in Akron, Ohio. If you are an NBA fan, you have probably heard of Akron. There is a local kid who has done pretty well from the NBA.  I went to school about 10 minutes from where I live. I work at Test Double. You may have heard of Test Double. Justin Searles, one of our founders.  It was on the program committee for Rails Coffee. Test Double, our mission is to improve the way the world builds software. I know that sounds very audacious, but we truly believe that every programmer has it in themselves to do that.  I believe every person here has it in themselves to do that, and that is why you are here. Definitely it has been a great company to work for, and I am very proud to represent Test Double here.  Personally, one thing I have done, my biggest claim to fame lately is I created a Ruby gem called Concurrent Ruby. You may have heard of Concurrent Ruby because it is used in some very well-known projects like Rails.  Concurrent Ruby is a dependency of Action Cable. In Rails 4 and 5, it is used by Sprockets. I also use some gems like Sidekick, Sucker Punch, elastic search and Log Stash Utilities.  Microsoft is your Ruby tool. Much of what I am going to be talking about today draws my experience from that, but this is not going to be a sales pitch for that. This is going to be about ActiveJob and Rails itself.  Because this is beyond the magic track, this is not going to be an introductory topic. This is going to be a deep dive into the internals of ActiveJob. I have had to make a couple of assumptions in doing this.  I am assuming that if you are here, you have used ActiveJob, probably in production, you have used one of the supported job processors, you have some understanding of concurrency and parallelism. If you need a better introduction to ActiveJob itself, I highly recommend the Rails guides.  The Rails guides are excellent at this and provide a lot of great information. If you need an introduction to concurrency within Ruby itself, same-less plug. I did give a presentation last fall at RubyConf called Everything You Know About the GIL is Wrong.  That video is available on YouTube and that could be an introduction into that.  So with that, let's jump into what is ActiveJob. So in order to get into the internals of this, I need to briefly remind us of what it is and where it came from.  So ActiveJob, according to Rails guides, the definition is this. ActiveJob is a framework for declaring jobs and making them run on a variety of queuing back ends.  So ActiveJob can be everything from regularly scheduled cleanups to billing charges, mailings, anything that can be chopped up in small units of work and run in parallel. The couple key terms there. It's a framework.  We're going to talk more about this, but asynchronous job processing, pre-existed ActiveJob.  There were things like Backburner Delay Job, Q Rescue Sidekick, Sneaker, Sucker Punch. Many of these things existed before ActiveJob was created and ActiveJob came along as a way of unifying those.  ActiveJob helps us schedule tasks to be run later. That was mentioned briefly this morning in the keynote that when you don't want to block the currently running PREB request and you want something to happen later, you use ActiveJob in order to make that happen.  And that can happen through what we call ASAP processing, which is where you get to this as soon as you can or by scheduling it at a later date and time, potentially.  And this is also good to support full parallelism. Some of the job processors are multi-threaded. Many of them, however, actually are forked. I'll talk about more and can run multiple processes on machine and scale across multiple processors.  And in some cases across multiple machines.  So the impetus for ActiveJob is that background job processors exist to solve a problem. We have these long-running tasks that we don't want to block the web request.  So we want to be able to send a response back to the user and get the page rendered for them. And some of these tasks then occur after that.  So for example, if I'm sending an email, this email takes time. It's a sin to begin with. Why should I block the web request to make sure that email posts when I can send the response back and have that post shortly thereafter?  So ActiveJob supports that. And the processors behind that support that.  So like I said, ActiveJob, this is going to be important when we get into the internals, ActiveJob came later.  And there are all of these job processors. Each one was unique. And each one, they all did virtually the same thing. They had slightly different capabilities.  And when it got to a different late, they all saw the same problem.  Right. So ActiveJob was created to provide a common abstraction layer over those processors that allow the Rails developer to not worry about the specific implementation.  Right. This sounds familiar. This is not dissimilar to what ActiveRecord does. All right. Relational databases existed. ActiveRecord created an abstraction layer over that that allows us to run, use different databases to fairly  switch between different databases if necessary. Most importantly, run different databases and test, prod, and dev. Right. ActiveJob does the same thing.  It allows the abstraction layer that will allow us to choose different processors, change different processors as our needs occur, and run different processors and test, development, and production.  Right. And so ActiveJob had to do that while supporting the existing tools that people were already using.  So, according to RailsGuides, is picking your key back end becomes more of an operational concern. Use the developer, don't care which back end you use is being used.  You simply write the jobs and let the application and then use whichever back end in whatever environment makes the most sense.  So, because we're looking at some code, I want to real briefly remind us what the code looks like for ActiveJob before we jump into the internal.  So, this is a simple job class. This should look familiar to everybody. The important things are that this class extends ActiveJob base and that it has a method called perform.  Right. Most of what ActiveJob does is encapsulate it in the ActiveJob base class, which goes and will eventually, as we look through the details, call this perform method on your, on object of this class when the job actually runs.  And look at those details. Right. And as a reminder, the way we configure our back end is we use this JobQ, ActiveJobQAdapter configuration option within our application RB.  Now, InsideJob is what I'm going to call the adapter we're going to build here.  We're actually going to build in here a real adapter that is functional.  Right. So, the, all of the adapters that are supported by Rails have a symbol that follows normal Rails inflections that maps the adapter name to what you set the convict value for.  So, if InsideJob existed as a supported adapter in Rails, this would be how you would set that.  Right. Then, that's how you configure which back end you want to use. And then later when you want to actually do something later, you call the perform later method on your class, passing it one or more parameters.  Okay. And that should look familiar to everybody. And if you want to schedule the job for a certain time, then you can use the set function to specify when and there's a number of different ways you can do that.  So, that's just a reminder of what we see on the front of ActiveJob. All of that should look familiar to everybody.  We're going to talk about is what goes on behind that when you make this perform later call. Okay.  So, I guess we're going to build a asynchronous back end here, right up here, during this presentation.  One that actually works and is functional and will meet a, it's minimal, but it will meet the requirements of ActiveJob and show us how this works.  So, a couple of things just to give a sense of where we're coming from.  Like I mentioned, there are multi-threaded adapters and there are forked adapters.  Multi-threaded adapters run your job in the same process as the Rails app itself. Okay. There are a couple that do that.  The advantage of that is those can be very fast and you don't have to spawn separate processes and manage separate processes.  All right. We all know that MRI Ruby does have some constraints with concern to concurrency, but it's not as bad as most people think.  That's what I talked about at RailsConf last fall. And since most, MRI Ruby is very good at multi-threaded operations when you're doing blocking I.O.  And most of the tasks we're going to be making these background jobs for are doing blocking I.O.  They're sending emails. They're posting things to other APIs.  And so, since they tend to do blocking I.O., they tend to work very well with Ruby's concurrency model.  So, a threaded back end is simpler because we don't have to manage separate processes.  Many of them, however, do use or they do spawn fork separate processes where you have to run separate worker processes.  Those give you full parallelism, but they require active management of those processes.  So, for what we're going to build here, we're just going to do a multi-threaded one because I can do that very easily.  And it will demonstrate all the things we're going to do. And we're going to use thread pools for that.  Most job processors will also persist the job data into some sort of data store.  Redis is very popular for this. The reason for doing that is that if your Rails process exits, either on purpose or by crashing,  if all of your job data is in memory, you're going to lose it and those jobs will never run.  So generally speaking, for production, you want to have a job processor that does store the job data in some sort of external data store  to allow it to persist beyond restarts. We're not going to do that here mainly because in simplicity,  I want to demonstrate what goes on in active job. We don't have to go to that level of effort.  So, our job processor will not persist through a data store. So it makes it good for testing and development.  But not necessarily we wouldn't use what I'm going to build here today for production.  So in order to do this, we're going to need three pieces.  The first one is active job core. This is provided by active job itself and it is the job metadata.  I want to talk about this more, but it is the thing that defines the job that you need to perform later on.  It is probably the, I'd say the most important piece of all of this because it's the glue that binds everything else together.  The two pieces we're going to build today are the QAdapter and the Job Runner.  Remember, active job came about after the Job Runners. So the Job Runner is independent and it provides the asynchronous behavior.  The Job Runner actually exists as a separate thing. Sidekick is a separate thing.  Sucker Punch is a separate thing. You install those separately.  The QAdapter has the, its only responsibility is to marshal the job data into the asynchronous job processor.  So the job processor provides the asynchronous behavior and the QAdapter marshals between your Rails app and that job processor.  And those are the two pieces we're going to build here today, the QAdapter and the Job Runner.  For all of the Job Runners supported by Rails, the QAdapter is actually in the Rails code base.  Okay. If you go into GitHub, go into the Rails code base and look at active job, you'll see that there's a folder of QAdapters  and there's one QAdapter in there for each of the processors that Rails supports.  There's also a set of unit tests as part of the Rails code base that will run against every one of these job processors on every commit.  And they ensure that all of the supported job processors meet the minimum requirements of active job.  The one we're going to build today actually will pass that test suite.  And once it's in the job, it will pass that test suite.  So strictly speaking, the Rails core team has responsibility for the QAdapters and for that test suite.  But knowing from experience, the people who create the Job Runners themselves work very closely with Rails to make sure that those adapters are up to date and work well with the processors.  So let's stop then and talk about the active job core class.  Like you said, this is the glue that ties it all together.  It's not obvious.  So this is the job metadata.  It is an object that represents all of the information about the job you've posed.  It carries with it the proc that needs to be run.  It carries with it things like the Q and the priority was stopped in a minute.  And it carries with it all of that metadata.  It provides two very important methods which I'll talk more in a minute.  But they are the serialized and deserialized methods.  These are very, very critical.  I'll talk about them in a minute.  The job metadata itself, there are several attributes on this object which we will look at and use internally within active job.  These are not things that you as a Rails developer have to know about, but these are things that when turned inside of active job are very important.  One of them is the Q name.  Most of us should be familiar with that.  You can specify when you create a postman these jobs what Q should run against.  And if you don't specify, it's the default Q.  Priority, some job processes support prioritization where higher-party jobs run first.  We're not going to support prioritization in ours.  That's optional.  But the priority would be attached to this as well.  If you schedule a job to run at a specific time, you get an attribute called schedule that which tells you when.  We'll look at that because we are going to do scheduled jobs.  The job ID is internal to Rails.  And there's a unique ID within the Rails instance itself that identifies each job.  Rails uses that within active job to track each one of these.  The provider job ID is one that you can provide within your job processor.  So if we want to do it within our job process, if we want to have our own kind of ID system that made sense for us and worked,  we can then attach it to the job metadata under the provider job ID.  So Rails does not create that.  We would create that ourselves.  We're not going to use the provider job ID today because it's not essential,  but it is available and it's something we would add.  So let's actually build a QAdapter.  We're going to go outside in.  So like I said, the QAdapter is responsible for marshaling data into the job processor.  The job processor is the more interesting piece.  We'll look at that in a minute.  But we're going to start with the QAdapter.  And we're sort of going to pseudo-TDD this.  The QAdapter, most of the QAdapters were written when active job was created because the job processors already existed.  And they had to handle that marshaling.  In our case, because we don't have a QAdapter,  or excuse me, we don't have a processor yet,  we can decide what the API is going to look like.  So within our QAdapter, we only need two methods.  It's very simple.  One is NQ and the other is NQAt.  The NQ method takes that job object we looked at a minute ago,  and it marshals that into our processor.  And the NQAt takes the job and a timestamp and marshals that into our job processor.  So notice in this case, I've decided to make the API very simple.  We're going to create a thing called inside job.  We're going to have class methods NQ and NQAt.  We're going to pass the serialized job.  We're going to pass the QName.  And in the case of the NQAt, we're going to pass the timestamp.  So a couple of things to note.  One, this is not very OO.  These are class level methods that we're calling on this class.  And I did that because I want to emphasize the stateless nature of this.  This is very critical to understand.  Active job is by its nature stateless.  The state for your job is encapsulated in that job object.  All of the metadata about the job, everything related to that job,  all of your state is in that that we're passing through.  The actual QAdapter itself is inherently stateless.  Its job is just to...  In your notice, we even call a class level method when we post the job.  Because we're sending this thing to happen later on.  It's a fire and forget.  We're not creating anything that's going to be persisted.  And in fact, any kind of stateful behavior here would be potentially thread unsafe.  So we're just going to call these class methods and throw this data at it.  And then we'll build those class methods in a minute.  And that's all it really takes to build a QAdapter.  Now, one thing that's very important here is the serialized method.  And I have to go into this in a little detail.  The reason why we call this serialized method is twofold.  First off, and less importantly, is thread safety.  Remember, Ruby is a shared memory language that has object references.  So if we have maintained a reference to anything that was passed into that,  and we hold onto that reference when this thing finally goes and gets processed later on,  if it's processed in the same process on another thread,  we run it a potentially not thread safe behavior.  Now, the normal usage pattern makes that not really a big deal.  But if we serialize the job into a representation of that,  we then let go of those references and make it thread safe.  There's one important reason, though.  And the most important reason is for consistency.  Remember, we want to be able to work with multiple job processors in prod and dev  and even test we want to be able to.  So when those job processors are going to persist into a data store such as Redis,  they must serialize somehow.  Maybe this can't take a Ruby object and throw it into Redis.  Or throw it into a traditional database, we have to serialize it somehow.  That's sounding bad.  So if every job processor created its own serialization method,  we could potentially run into problems when we switch between these.  We don't want to have hidden errors where we run this in tests and we run it in dev,  and all the serialization works,  and then we run it in production with a different processor  and the serialization fails or does something different.  So ActiveJob provides one common serialization routine method and one deserialization method  so all of the job processors can choose to serialize the same way.  And in so doing, that will allow make sure we reduce one potential set of errors  when we move between job processors.  So we are going to serialize here, even though, you know, this is the simplified version  and we're not storing this in a data store.  We want to do that serialization to make sure we get that consistency across processors.  So internally, like we said, we need to do two things.  We need to provide inside the QAdapter, I'm sorry, I moved on to the job processor now.  So we have the QAdapter, now we need the job processor.  The job processor's responsibility is to provide the asynchronous behavior  and that asynchronous behavior is QDependent.  So we want to have multiple Qs and have each Q process a different set of jobs.  So for this, we're just going to use a, what we need is we need to be able to post jobs  into different Qs and have them behave asynchronously.  We're going to use a simple thread pool for this, right?  Because within the context of this simplified application, a thread pool works fine.  A thread pool has its own Q and therefore, by creating a separate thread pool for each Q,  we do get a separate Q for these different jobs.  We just have to map the thread pool to the Q name, which we'll see in a minute.  And then, obviously, a thread pool has one or more threads and therefore provides asynchronous behavior.  So we can very simply deal with these needs of the Q and asynchronous behavior by just creating a thread pool.  Okay?  So what we're going to do here is we're going to create the thread pool,  but because this is all very multithreaded and therefore needs to be thread safe,  not only are we creating these threads within our job processor itself,  but because Rails can be run under multithreaded web servers,  we need to go through a couple of hoops in order to get some thread safety here.  So we're going to use a concurrent map class.  This is similar to a Ruby hash and support similar APIs,  but it has some additional behaviors.  One, it's thread safe, but it also has some additional behaviors to make that work.  Hopefully, most of you know that with Ruby's hash, when you create a new hash,  you can pass a block to the initializer and that block will be called if the key does not exist.  And that block will then be to initialize that key.  So what we're going to do is whenever we try and retrieve a thread pool from our map of Qs,  if it doesn't exist, we're going to create a new thread pool at that time.  So we'll lazily create our thread pools as new Qs are needed.  Basically do this together and just see kind of one way that you might make this pass.  So this compute of absent is just a necessary thing in order to provide the  atomicity and synchronicity that we need to have in order to create this new thread pool in a thread safe manner.  So there's some concurrency needs there, but the end result is that's basically like creating a hash  and passing a block in to the constructor.  And then we're going to have a create thread pool class.  In this case, we're just going to create a cache thread pool.  A cache thread pool is the simplest kind of thread pool we can create rather than getting into the details  of all the different configurations we could do.  Basically a cache thread pool has an unlimited Q size.  It will grow and add more threads as needed.  And with threads become idle, it will shut them down and remove them.  So over time, we'll get an optimal number of threads in that, which for our simplified processor is fine.  So I mentioned that we need an NQ method inside our job processor.  It's going to look like this.  Basically, when we NQ this job, more jobs NQ, we're going to simply post  the job to the thread pool.  And when the thread pool pulls it off, we're going to call active job base execute.  That's the important part right there, active job base execute.  The first line, the Qs, the posts, that's just getting the thread pool, creating a new one if necessary,  then posting that to be run by the thread pool whenever the thread pool has an available thread.  Active job base execute is responsible for actually interrogating the job,  looking up our specific class to process that job,  and then posting and calling the perform method on an instance of that and passing in the arguments.  So when you, in your class, early on we saw we create that perform method and it takes a set of arguments  and it runs, active job base handles the interrogation of the job, creating an instance of that  and calling that method.  All we need to do is call the execute on that in our, when our thread pool takes this and runs it later on.  And that's all it takes.  Active job handles that, like I said, the internals of that.  And that right there is enough for us to actually post asynchronous jobs that are performing in ASAP way  in a real environment.  Now for the NQ for later, it's a little more complicated.  Right now.  We have to get into the time.  You're going to commit.  So fortunately we do have at our disposal a high level abstraction that handles these kinds of scheduled tasks.  And, coincidentally, it's called schedule task.  All right.  So the internals of schedule task are sort of beyond the scope of this,  but the idea is a schedule task will take a number of seconds in the future that something is supposed to occur  and it will queue it up and it will add that roughly that time.  It will then pass it on to a thread pool to make it work.  Where we are.  So when we notice when we actually use our perform at and we use that set method, right, Rails provides a lot of convenience things  for allowing us to specify when the job happens in the future.  Right.  Rails gives us all those great time helpers that we like, you know, one day from now and, you know, one week from now  and at certain times and so forth.  Active job is responsible for taking all of those convenience things that we use as Rails developers  and converting them into a number of seconds in the future when this runs.  Okay.  So by time we get this, we already have the number of seconds in the future.  So in our job process, you don't have to worry about all of those wonderful date utilities that Rails has.  Rails does that for us.  Right.  So in this case, it's really convenient for us because schedule task, not coincidentally,  takes just a list, a number of seconds in the future when the thing should run.  And in this case, normally within concurrent Ruby, all of the high level abstractions run on global thread pools.  So you don't have to worry about managing your thread pools.  In fact, most developers should never use a thread pool directly.  All right.  Most libraries that provide thread pools provide them internally and provide high level abstractions that allow, that use those thread pools.  So a normal circumstances, a schedule task or a future promise, any of these things, would use the global thread pool.  But in this case, we need a specific thread pool because that thread pool represents our queue.  Fine.  I'm just going to add.  So all of the high level abstractions in concurrent Ruby support the tendency injection of a thread pool.  So in this case, this executor option, which is very common, is a way of saying, when you do run this thing, run it on this specific thread pool.  So.  So what we're doing here is saying, look, we know how many seconds in the future this thing needs to run.  We know which thread pool we want it to run on.  Just go and handle that.  Schedule task handles that.  And at the time the thing needs to run, it'll then grab that job and it will run this block.  And we're going to do the same thing we did before.  It's just called active job base.execute.  That execute method doesn't know anything about the sinker's behavior.  It just knows now is the time to execute that.  Same thing we saw a minute ago.  And just in case we somehow get a time value that's not in the future, we're going to just check that delay and we're going to post it directly if somebody's not in the future.  And again, that's all it takes to post the task later on.  Rails handles all the time sensitive stuff.  We just need to make sure that we can do it at that time in the future.  And believe it or not, that in its entirety is a functional asynchronous job process.  So the next slide doesn't have a bunch more code on it that we normally see.  And I'm putting this on one slide because I want you to see just how simple this can actually be.  That in fact a real functioning asynchronous job processor can in fact fit on one slide.  And this is basically it.  We have a class called inside job.  We have our Qs constant where we have this thread safe map where we're going to keep track of all of our thread pools.  We're going to have that create thread pool method which will just return the pool we want.  Then we have our NQ behavior which just throws the job onto the thread pool.  And then we have NQ at which actually looks at that delay and that time stamp and gives it to a scheduled task.  And that right there is actually a fully functioning asynchronous job processor that plugs in that can work with active job.  And like I said, the other part was the QAdapter.  And remember the QAdapter just looks like this, right?  It was simply when active job calls NQ or NQ at simply post this thing off into my job processor.  So that's it.  So believe it or not, that is actually a fully functional asynchronous job processor that will work with active job and could be used in test or development in order to actually get asynchronous behavior without having to install registers under deep dependencies.  So the next question you're probably going to ask is, all right, Jerry, are you going to put this code up online so we can look at it like this?  And the answer is yes.  If you want to see this code, you can find it in a very convenient place and that's Rails.  The genesis of this presentation was that last fall I went to the Rails team and said, you know, it would be really useful if we had a simple asynchronous job processor in Rails 5.  As you all know, we can in our config specify the inline adapter.  The inline adapter will go and it will run the job synchronously so we don't have to deal with those underneath dependencies.  But the problem with that is it's not real asynchronous behavior.  And if we're using the inline adapter in test or development, we can sometimes mask problems by not having real asynchronous behavior.  They said, why don't we just build a simple one and we'll call it async job. We'll make the, instead of inside job, it'll be async job, we'll make the symbol just async.  And why don't we allow people in test and dev to run these jobs really asynchronously in order to potentially find bugs in them.  And the Rails team said, that's a really good idea.  And they worked with me and we got this merged into Rails 5 last fall.  So if you use Rails 5 and you use the async processor, this is basically what you're going to have.  This code was lifted almost line by line from the original implementation of that.  Now, since then, the Rails team has done some refactoring on that.  So if you go and look at the implementation now, it'll look a little bit differently.  So just to give you some context of what you see different, they decided to collapse things into one file.  When I originally wrote this, I had two files, one for the QAdapter and one for the job processor to sort of mirror that normal behavior you would have of the QAdapter  and the processor being separate.  They collapsed them into one file because they're very short and didn't need to be two.  They renamed some stuff to go along with better Rails conventions.  They are assigning that provider job ID, again, in this case, we're not really needing it,  but having that does, again, provide for greater consistency with the production ones.  And they decided to throw one thread pool for everything and it's fenced with having multiple queues.  Because, again, in test, all we care that these things happen asynchronously.  We don't particularly care about configuring the queues for various different behaviors.  So if you go look at it now, you'll still see async job and it will do exactly what we showed before and it's right now available in Rails 5.  So if I've piqued your interest and you want to learn more about this and see other things that you can do with this,  the two things I would suggest you look at more deeply are Sucker Punch and Sidekick.  Sucker Punch is a threaded in-memory async, just a job processor.  It does a lot of what this does, but does it way better and more fully.  The creator of this tells me that the main use case is if you want to send emails from a hosting provider,  a one-click hosting provider like Heroku, you can fire off these emails because there's not a lot of high cost of failure if that thing goes down.  And so those jobs are just retained in memory, not persisted through Datastore.  But Sucker Punch does use thread pools just like this does.  It does map those queue names to thread pools and provides some configuration of those.  And also does some really cool things where it decorates every job with a job runner class that does certain cool things like track the number of successful jobs,  track the number of failed jobs, handle errors, and do things in this nature.  So it's a really good example of how you can decorate a job when you push it into a thread pool and do some really cool things.  Like I said, for most of us, we shouldn't use thread pools directly.  The high level abstractions in the concurrency libraries provide those capabilities.  But this is a really good example of how you can do that.  And also, Sucker Punch does some really cool shutdown behavior where if the Rails app is shut down for some reason,  it will look at the number of the jobs that are still running and try and allow the jobs to execute completely before shutting down and some other things.  So there's some cool stuff in there.  Sucker Punch uses a lot of the tools that we saw in here.  It uses concur Ruby thread pools.  It uses concur Ruby's scheduled task.  Another great one is, of course, Sidekick.  Sidekick is also an in-memory, I'm assuming, it's also a multi-threaded job processor.  It does not use, it does persist all your stuff to a data store so that way your job data will persist beyond a restart of the application.  It does not use thread pools the way we saw here. Sidekick actually spawns its own threads and manages its own threads,  but it still deals with all those same things with the internals of an active job.  And of course, Sidekick has a whole bunch of additional features.  Like I said, Sidekick doesn't use concur Ruby's thread pools,  but it does use concur Ruby for some of the low level synchronization and atomicity stuff, thread safety stuff you saw here.  So there's two great examples.  If you want to look at this further, then go look at those code bases and see beyond what we've done here.  So with that, I just want to say, again, I work for TestDouble.  We are hiring, and we are also for hire.  So if you, we love talking to people about software development and about software and about how we can all improve software.  So if you'd love to chat with us, by all means, reach out to us.  You can find us on email, social media.  Myself and Justin will be here for the rest of the conference.  Like Justin will be speaking on Thursday in the afternoon at 3.30 in, he's going to be talking about RSpec in Rails 5.  So I've got stickers up here.  I've got stickers in my bag.  I hope you had a chance to talk with you sometime before the conference is over.  And with that, again, my name is Jerry DeAntonio.  Thank you for having me.  So I do have five minutes if anybody has questions I see or don't want them to run out.  That's cool. I'm hungry too.  So the question was resource contention within the job itself.  If you have multiple threads running simultaneously and trying to do things, all of the asynchronous behaviors provided by the job processor itself.  So all active job does is provide the compatibility layer.  It's important that the job processors themselves handle all of the concurrency, any kind of locking or synchronization that is necessary.  But generally speaking, if you follow the best practices, a lot of that contention goes away.  So for example, you're not passing an active record object.  I did it.  You're passing an ID, which you can then use to pull that up later on.  We're serializing the jobs so that we're not storing references.  But ultimately it is up to the job processor itself to be thread safe.  So the question was, would you be able to use multiple job processors simultaneously?  And the answer is yes, but not through active job.  Active job only allows you to specify one handler.  However, all, as far as I know, all, I'll say most, but as far as I know, it's all of the job processors can be used outside the context of active job.  So for example, you might specify sidekick for being your main job processor, but say for certain things you want to use sucker punch,  you would then just instant sucker punch directly.  And so you can do it that way.  And I don't know, I can't imagine why Rails would change that.  But again, it's very possible.  Yeah.  So the question was, could we subclass active job and have two different runners?  I guess, again, it's ruby.  We could probably do anything we want.  But there's that one configuration value within the application config.  I guess we could specify, of course, we can create our own configuration values.  We could create some new ones, grab those and do something of that nature.  I'm sure it would be possible, but it's not something that would be directly or easily supported by Rails in active job itself.  So multi-threading is just when you have multiple threads, right?  A thread pool.  So the question was, difference between multi-threading in general and a thread pool.  A thread pool is a managed thing where the queue and all of the threads are managed by the object itself.  So one of the things, like I said, so I could spawn my own threads just by calling thread.new, right?  What happens if those crash? What happens if I want new threads?  What happens if I have idle threads?  How do I enqueue those things there?  There's a lot of plumbing involved in that.  We can always spawn multiple threads, but in order to manage that, there's a lot of extra stuff.  How do we handle exceptions?  If you throw an exception on a thread, it will crash the thread.  How do you handle that?  So a thread pool takes all of that, puts it in one object with some very well-known, very common cross-language algorithms and manages those things.  So you create a thread pool, you give it a set of configuration parameters, things like how many threads to run at a minimum, how many to maximum, how many things can you enqueue, if the queue gets full, what do you do, if you can't generate it.  The outbreak system won't give you more threads than what you do, and it handles all of that for you.  So all you do is just create this one abstraction, the thread pool, and you throw stuff at it, and it manages all of that in queuing and dequeuing.  How to find those migrations.  If threads crash, it will handle that and so forth.  So there is some overhead in the thread pool itself because of all of that.  But just like anything else, that overhead comes at the value of making you not worry about those things.  So generally speaking, you start with high-level abstractions that use the thread pools, so you don't have to worry about that.  Then maybe later on, you specify your own thread pools and inject them and take a better control.  And then maybe if you're this guy over there, you just write your own threading yourself.  But that's sort of the progression.  And that is a fantastic question.  The question is, how does it handle exceptions?  What we did here does not handle exceptions very well at all.  The thread pool itself will protect itself from any kind of exception on the thread.  So thread pool will not allow its threads to die because of exceptions.  Thread pool doesn't do much with them.  Again, this is one of the reasons to use that, to use a high-level abstraction.  Because if you use a future or promise or active agent, those things have consistent idiomatic ways.  Idiomatic ways of handling things like return values and errors and so forth.  So if you look at Sucker Punch, the job decorator class in there actually handles the exception.  It's like a nice little bit of Rails magic.  On the thread itself before it bubbles up and then doing things with it.  So again, the high-level abstractions are the ones where people provide you with better error handling and consistent idiomatic.  They're dealing with return values and minimizing the weight on things and so forth.  And again, that's why you should always start with high-level abstractions and then only inject the thread pool.  Thread pool is meant to be the very lowest level in that and just provides sort of the engine.  So like I said, the actual job processor itself will handle things like errors.  So if you look at whatever you use, which one of those supported job processors you use,  they are all doing the error handling in this case.  Because this is the bare bones minimum, I'm not handling errors at all.  Your job is just going to die, you'll not know about it.  But again, this is meant to be minimal and trivial.  But if you use any one of the full-blown production-ready job processors,  they will handle the decoration of that job and they will handle the errors  and they will have their own way of doing that.  That's one thing that ActiveJob does not include is a consistent way of handling errors.  So of course, you could always put your own error handling in your perform method  and handle it there, but ActiveJob doesn't really do that for you.  I wouldn't do one.  I mean, in terms of production, the ones that are out there are very fantastic.  They're very mature. They've been used a lot.  The reason for this one was to put it in Rails itself so that for development and testing,  you can run your tasks synchronously and get a better understanding of how they're going to work in production.  One thing, I've talked to a lot of people.  In fact, one of the...  If you go back and look at that commit and you look at the actual discussion around that PR,  it was actually DHH himself who said,  hey, I really like this idea.  I generally install Sucker Punch for dev and test.  It would be nice if I could just do that within Rails and not have to have that extra dependency.  Sucker Punch is great.  I've worked with Brandon, the creator of Sucker Punch, and he's fantastic.  But it's an extra dependency for just dev and test.  Rails have done a great job of providing the inline adapter for dev and test.  So it makes sense for Rails to provide that simple async one.  And so we just minimized what we actually need and put it in Rails itself.  So now for dev and test, you can do that and get a better sense of the real behavior in production, what it might be.  And how about the...  I'm sorry?  It's in Rails 5 now.  So all you need to do is in Rails 5 is just say async in your config and it's there.  Anything else?  All right, thank you very much.
Inside_ActiveJob_0_47.png Inside_ActiveJob_48_119.png Inside_ActiveJob_120_227.png Inside_ActiveJob_228_3815.png Inside_ActiveJob_3816_6287.png Inside_ActiveJob_6288_12299.png Inside_ActiveJob_12300_19151.png Inside_ActiveJob_19152_19931.png Inside_ActiveJob_19932_30395.png Inside_ActiveJob_30396_44183.png Inside_ActiveJob_44184_45875.png Inside_ActiveJob_45876_50879.png Inside_ActiveJob_50880_50939.png Inside_ActiveJob_50940_50999.png Inside_ActiveJob_51000_51071.png Inside_ActiveJob_51072_51131.png Inside_ActiveJob_51132_51203.png Inside_ActiveJob_51204_51251.png Inside_ActiveJob_51252_51299.png Inside_ActiveJob_51300_51359.png Inside_ActiveJob_51360_51467.png Inside_ActiveJob_51468_51635.png Inside_ActiveJob_51636_51767.png Inside_ActiveJob_51768_51815.png Inside_ActiveJob_51816_51875.png Inside_ActiveJob_51876_52103.png Inside_ActiveJob_52104_52283.png Inside_ActiveJob_52284_52391.png Inside_ActiveJob_52392_52511.png Inside_ActiveJob_52512_52595.png Inside_ActiveJob_52596_53435.png Inside_ActiveJob_53436_53495.png Inside_ActiveJob_53496_53567.png Inside_ActiveJob_53568_53903.png Inside_ActiveJob_53904_53951.png Inside_ActiveJob_53952_54275.png Inside_ActiveJob_54276_56459.png Inside_ActiveJob_56460_57395.png Inside_ActiveJob_57396_57491.png Inside_ActiveJob_57492_57743.png Inside_ActiveJob_57744_58223.png Inside_ActiveJob_58224_59351.png Inside_ActiveJob_59352_59855.png Inside_ActiveJob_59856_60947.png Inside_ActiveJob_60948_60995.png Inside_ActiveJob_60996_61103.png Inside_ActiveJob_61104_61247.png Inside_ActiveJob_61248_61252.png