Breaking Bad - What happens when you defy conventions?
CC Attribution - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Gockel, Christoph
With Rails being over ten years old now, we know that the Rails way works well. It's battle tested and successful. But not all problems we try to solve fit into its idea on how our application should be structured. Come along to find out what happens when you don't want to have an app directory anymore. We will see what is needed in order to fight parts of the Rails convention and if it's worth it.
 Welcome everybody and thanks for coming. Today I'm going to talk a little bit about Rails and how awesome it is. And I think part of why it is so awesome and why it is great is its community and the huge ecosystem around it. And when I try to explain Rails to newcomers to the industry or even colleagues, I think of Rails about this city map, this huge massive city map. It's big and consists of different areas like over there is the active record district and there is the initializer town and all these things. And in our day-to-day work, it can feel kind of intimidating sometimes even. You're surrounded by all this functionality and you don't know what's happening and you feel constrained sometimes even. And while it's true that Rails has its certain ways and patterns how things should be done, this does not mean that we cannot find or create our own little areas of freedom. And today I'm going to talk about some of the areas we explored at AtheLite in order to help us work in a more efficient way with Rails. So at the beginning I want to talk a little bit about the status quo. Then we talk a little bit about breaking of some things followed by a quick wrap up about general application architecture and then talking about some of the trade-offs. So quick question to the round, who here knows status quo? A couple of hands? All right, cool. So in contrast to Rails, status quo had a very simplistic approach. They always only played with three chords. And while I'm not going to continue today to talk about the band status quo, I thought coming here from London, I should at least take up a British reference. So we're done with that. We continue with the actual status quo as in the current state of affairs. Two years ago, DHAH mentioned at RailsConf that Rails for him meant that he was going to use Rails for him. He still wants to be able to use Rails in order to write web applications. And I think this is a really cool feature and idea behind it because everything comes with Rails. Like batteries included, you can continue or you can start writing web applications without the need of anything else. I'm more of a ultra-light backpacking person myself. And while I'm not prepared for the zombie apocalypse, I always have just enough gear with me for my current adventure. And this is reflected in the Pareto principle, mostly known also as the 80-20 rule. So for example, 80% of your users use only 20% of your product's functionality. Or in Rails' case, 80% of Rails applications out there use only 20% of what Rails offers as features. And the background of this talk has something to do with Omakasa Rails applications. And throughout several years, and having done a couple of Rails projects, we felt the same pain over and over again. And that is that Omakasa Rails applications lead to slow test suites. And this is a problem for us at Aethlite because we are used to quick TDD cycles. We want to be able to write a little code, run the test suite, to get feedback immediately. So this writing a little code and running the test happens, I don't know, five to six times a minute. Sometimes if I'm typing sloppy ten times. And I want this fast and quick feedback loop. Unfortunately, the bigger your Rails code base grows, if you follow the regular Omakasa style, you lose out on these things. And in a previous application where we felt the pain and it was the last application we built in this way, this will be our reference application for throughout this talk. And we ended the project with around about 4,800 tests and the full test suite ran in about six minutes. And waiting six minutes until you know whether refactoring worked or not is too long in my opinion. So I need my code base malleable in order to make a quick decision. Yep, okay, this refactoring worked, continue, this buggy fixed, or it's not fixed. I want to get this feedback really, really quick. So after having seen that throughout a couple of projects, we went back to the drawing board, put on our thinking ads and started to really dig into what is the actual issue. And this is me with my thinking add on. And one hotspot we identified was the Rails Buddha process, which looks roughly like this at the beginning, the Rails framework is loaded, then it will load all the dependencies from the.jamp file and then it kicks off its initialized bang process, which loads all the code from all configured Rails engine and so on and so forth. And during that process, something like called eager loading or in the docs it's called or referred to as eager loading kicks in. And eager loading requires or is responsible for requiring all code that is inside your app directory, for example. And we started to modify our configuration a little bit. And in a typical Rails application, you see in the application.rb, you see a line like this require Rails all, require the whole world and continue. So we changed that a little bit in order to only require the two Rails ties or the two frameworks we actually needed for this particular application, which was only action controller and sprockets for asset management. It turned out, which I learned after we did this change, that there's also already support for this kind of configuration by Rails new. If you run Rails new with a minus minus help flag, you get a list of all configuration options you can provide there. And it actually allows you to skip a couple of plugins already and you see there skip active record, skip action cable. If you don't need them, you can already tell Rails, hey, don't generate the new project with these dependencies because I don't need them. And I did run this command with a minus minus skip test because I usually use RSpec for my testing. So what Rails new generates then is an application.rb that looks like this. It doesn't require Rails all anymore, only Rails, which is the BAM minimum. And then it actually tells you, hey, pick the frameworks you want, pick and choose. And it commended out the test unit Rails type already for me because I was skipping that in the previous step. So that's good and well. The next thing after requiring the initial plugins, there's a line that will look like this. Bundler require Rails groups. In previous versions of Rails, it looked a little bit different. It required the default and then the Rails nth. It has been changed, I don't know in which version, but this tells Bundler to require all gems from your gem file for your current environment. The problem with that is that this line will add a linear load time to your boot process, which means the more gems you add, the more time will be spent when you load up your application. And this is unfortunately a hard fact and a hard truth we need to accept. At the moment, there is no way around that. So we continued. There are a couple of more settings we started to tweak and one of them is active support. Supports a bare flag in order to say, hey, don't load everything from active support, but only the dependencies that are actually needed in order to boot up Rails. And then there are two more settings, cache classes in order to keep a class cache. If a class has been loaded, keep it loaded, and then we disable specifically the dependency loading. And this is because at that time, we were big fans of a thing called screaming architecture, which I will go to, which we will come back to at a later point at the presentation, but as a quick heads up, screaming architecture, the idea behind that was that it should be obvious for a new person on your team that to know by just looking at the directory structure what kind of application is it. And if you take the Rails core base as an example, if you generate a new Rails core base, you see app in config dblib public and I don't know what domain is represented in this particular application. Just because I've seen Rails before, I have an idea that seems like a Rails app. And if you drill down a little bit in the app directory, there are even more sub-directories which do not really reveal the intent beyond the application itself. So what we started doing is we moved everything that was in app into the lib directory with a proper namespace. So in this example, there was a movie organization application. So there you can see there's an edit index and show ERB template. There is a movie model, movie repository, and also the movie controller. Like everything that used to be an app, we moved to lib. We deleted the app directory altogether. In order to really make sure that nothing will be eagerly or auto-loaded or looked up anywhere, there are a couple of more settings for the application.rb. We made sure, okay, eager load false, don't try to load anything, and we made sure don't even try to look up any path because we pretty much nulled them. So we booted up our Rails application, no errors whatsoever. Trying to hit the index page, blew up with an exception, uninitialized constant movies controller. So I mentioned before Rails expects everything to be loaded up front. This is why this pre-loading, or eager loading happens in first place. So now Rails can't find our application controller anymore. Luckily for us, there is a thing called lazy loading to the rescue for us. And what we did there, in the sector, you can see a call to active support, constantize. So at that time, Rails tries to constantize a controller name based on the route definition we provided there. So because Ruby is flexible and awesome, we monkey patched constantize. More specifically, this particular one line you see there, we wrap the existing constantize and say, okay, if you try to constantize a controller, actually require this controller as well. And this makes the trick, everything worked, we could serve requests, everything worked fine. But it had an interesting effect on our code base because now we needed to have explicit requires everywhere. And this was a constant decision within the team. But it's also not very Rails-y, if you want to say so. What I like about it, it makes it obvious how many dependencies my current file I'm looking at has. So I refer to this as a manual static analysis. If you open up a file and it has 20 require statements, it gives you at least a hint to, well, maybe this thing is doing too much. And then you can start thinking about extracting things or splitting it up in different ways. Maybe it's okay, but at least you can start having a conversation and think about, is this thing doing too many things? And with all these settings now configured, and we were able to start a new green field project, which will be our example one. So green field project, we had everything at our disposal. We could do whatever we want because we were in complete control of what we want to do and how we want to approach it. So we did all these configuration settings. And here's the existing or the previous application again with the six minute runtime of the suite. I was not part of the project when we kicked it off. I joined the project when there was already 5,500 specs that ran in 23 seconds. And while I don't have a percentage here, I think this is a pretty big increase, decrease. Additionally to that, with all the tweaks about how many files need to be loaded at the beginning, loading or testing one controller in isolation used to take around 18 seconds until you knew, okay, this controller works or it doesn't work. Now with the new approach and the new configuration settings, we only needed to wait two seconds, which I think is still pretty good. One caveat here, though, is we made also a choice to not use Active Record at all. We switched to the SQL gem and a repository pattern in that way, which you don't need to discard Active Record for that. It's mostly about that you can switch your implementations for your persistency depending on the environment you're in. So when we were testing, we had an in memory repository in order to help us gain more speed. In production, of course, we use a real repository that actually connected to our Postgres database. But we could still have used Active Record for that. It's just we switched to SQL. No particular reason. It was just the way we started working with it. It's worth mentioning here, though, that this adds double burden on us. We needed to maintain two separate versions for each repository. We needed to maintain a Postgres implementation as well as an in memory implementation. But the benefits for that was that we gained a lot of speed for our test suite. So it was worth for us investing into this double maintenance. The way we did that was with shared RSpec examples that ran for both implementations. So we actually only needed to write the test ones, but implemented twice, basically. The second example I have was a Rails application we took over from a new client. And that started with 220 gems and a liberal usage of Active Record. The application itself came with 2,600 tests out of which were 730 controller tests and the rest of them were not just plain or Ruby object tests. There were a bunch of Cappie Bar tests as well as tests that integrated or verified some elastic search behavior as well as verifying that the side gig job was kicked off. So it wasn't just controller and the regular Ruby world. And the full suite ran in around 8.5 minutes. So we thought, okay, let's optimize it our way in order to gain more speed. And running a single controller with the existing application with the require Rails all way took around 14 seconds in that case. After we did all the configuration changes and all the required statements because now we actually needed to add all the required statements manually back into the code base which weren't there before because it was a regular or MacArthur Rails application. We were only able to cut it down to nine seconds. Still a little bit better but not the benefit we had before. And for a non-controller spec it looked a little bit better. If we were running a single non-controller spec it took around six and a half seconds. And we were able to cut this down at least to around half a second. So there was more optimizations. There was a better optimization for us than with the controller test but we weren't able to just apply it throughout the whole code base because manually adding all required statements took a lot of effort. I did all the benchmarking for two controllers. And at the end of it I had like my Git status showed me I had 150 piles changed. So because it's a client project we can't build the client for refactoring that we wanted to do for refactoring sake. So we would need to find a way to slide it in as we add more features. So we couldn't just flip the switch and say yep now your application is faster and we have a better development experience. But we could do some projections. So around eight and a half minutes was the previous runtime for this week. And with the knowledge about like it's not just controller test and non-controller test for us we projected numbers that we were able to cut it down to at least five to six minutes which is still fairly long too long for my own taste but a little bit better than before. And with all the changes and all the explicit requires I know that not many people would agree with me. The main point I'm trying to make here is that launching a test suite should not be a daunting task. You should you want to be able to run your test suite multiple times a minute to get this fast feedback. And if you only take one thing away from this talk is split up your spec halvers. Split your spec halvers in a spec halver that has just that just loads our spec and maybe define some global test halvers that you want to be using throughout your test suite and a separate rail spec halver that requires the existing spec halver and also requires the rail environment. This way you have this is a really low hanging fruit in order to at least optimize all your non-rails test. Like if you just test the class that does not require anything from rails you get a huge speed benefit. Having said all of that let's talk a little bit about the rails way. So what does this actually mean? In Rails's doctrine we can read that there is a value convention over configuration. We made a very conscious decision in our team that we favor expliciteness over implicitness. I like to read explicit and boring code. Like magic code does not I'm not a you I don't really like it. That doesn't mean that I don't like rails but it's just I favor explicit code over knowing that okay something will be loaded and something will be done for me in the background. It's worth mentioning that conventions are heuristics. They are not rules. You can break them and if you learn something new you might even realize how much this convention we used to follow for three years. Maybe it's not a convention anymore. We should use something else and it's not a bad thing. This comes the 80-20 rule applies here again. Like all these conventions we find in rails work very very well for 80 percent of rails apps out there but the remaining 20 percent might need something else and that is fine. It's not bad. It doesn't mean that rails is not the right fit for them. It's just that rails still offers enough flexibility in order to support the remaining 20 percent. You just need to maybe work a little bit harder to reap these benefits. As I mentioned earlier I want to talk a little bit about general architecture as well and while I'm not I don't want to bore you with 90s 70s material about what software architectures should look like there are a couple more recent architectures out there. First one is clean architecture which defines some entities and the core of the application that will be orchestrated by so-called use case implementations and then these use cases will be exposed through controllers in our rails application for example through the web. In a similar vein there is a thing called hexagonal architecture coined by Alistair Coburn which goes in a similar direction like at the core of your application is your actual application and you provide different adapters for different clients. Like you can have a GUI adapter in order to connect the QT GUI to it or an HTTP adapter to connect maybe a browser to it like think of rails here and these adapters do not only work for like inbound connections also for outbound connections like you have an adapter in order to connect to your persistence layer Postgres on MySQL or you have an adapter for the Arrest adapter that connects to Salesforce for example and last but not least in Martin Fowler's book Patterns of Enterprise Application Architectures this is a thing called service layer same idea again you have a user interface that connects to service layer which guards your domain model which then it itself has access to this it's called data source layer here it's your database at the end of the day and when we look at these three architectures they look very very similar they all have the same idea behind them and it's not something completely new even because if we take the service layer as an example and we focus on one particular area and we zoom in on that it's pretty much a layered architecture again like nothing new to learn here we're back in the future back in the 90s it's fine like new words for old concepts like isolate your application from the outside world that's pretty much it it's important to remember though that Rails is not your application and this is why I was emphasizing on the architectures a little bit and I want to show you an example of a pictures e-commerce platform here again in e-commerce platform we might have a user class that if user logs in we have a customer class representing this particular user and the user can have a basket a basket can consist of a couple of products and each product has a category and while we're checking out we need to select a delivery method as well as a payment method and at the end of the payment process we get an invoice at a PDF download for example and this is our core application like we don't see a controller there we don't see a session hash or any gnarly rails of all web details like this is the core of our application and this goes in the direction of the manager of design a book I highly recommend to read and in order to tie that back to the architecture we saw before let's have a look how that applies to our res application how that would apply to our res application so if we start from top to bottom our user interface is usually a browser and this browser connects through HTTP to our res application or our checkout controller and the checkout controller then translates this web request to a thing called our checkout which is our use case implementation in this instance and the checkout process the checkout use case then mediates between several domain model objects which ideally should just be plain old Ruby objects that define some behavior and then at the end it will create an order which is or which acts as our gateway to our database and this is pretty much it like top down one request comes in and you might have something new in the database how does that tie back to Ray as this MVC idea MVC has become this more of an abstract concept than a design pattern or anything like that and we've seen that a couple years back in the reds community even where there was talk about oh yeah we should pay for fat models over two skinny controllers until we realized okay these models get really really hard to test so we went to the other extreme and said okay let's keep the models really thin and put all the behavior in the controllers which made it really hard to test the controllers and it's not enough to only think or talk about these three buckets because I see this as a balloon that you just squeeze on one end you're not magically removing air or responsibilities from from things and you're just squeezing it here with and it will expand the balloon on the other side so it's important to not think about or focus on skin or fat anything we should focus on a healthy everything because spreading the responsibilities into small chunks will help us make create an easier to understand code base like if we only worry about classes that have five to ten lines it's really really it should be easier to understand than following a 50 method long method for example and when I was preparing this talk I had a chat with a friend over coffee and I was going through the topics and areas I want to discuss with him and he was asking me okay so we did this type of rails style for a couple of years now and he was asking would you do it again would you still do all your applications like this and I was thinking a moment and I thought well all the pre loading and loading optimizations yes for sure but the idea with the screaming architecture like removing the app directory and moving everything into lib probably not I think this is utterly overrated because if we think back to the default directory structure I don't necessarily need to know that this is a movie organization database or e-commerce platform for flowers or something like that it's okay to start with okay it's a res application if I want to know more I probably look into lib or if there's a source directory there's source and then I have my nice namespace responsibilities laid out there and with the separation of concerns or like separating your application from everything that is rail specific I actually like the idea to have these physically separated now like my core application is in lip nothing rails specific should leak into lib everything rails and web related should stay in app like this makes it not trivial but at least easier to upgrade rails as well because we don't need to worry about anything that is in lip maybe we wrap some active job dependencies inside lip but this is really about it lip is our application nothing of rails should leak in there so everything comes with a tradeoff right I'm not trying to sell you the promised land here like it's not that magically all of a sudden we will end up writing more and more awesome rails apps there are tradeoffs and there are hard tradeoffs you make for example explicit requires this is something that is not necessarily used in a typical rails application and if you take back the city example or the city metaphor at the beginning like we put up some big construction sites and just cut through a couple of blocks not without thinking but it we need to be aware that it comes with the cost because we need to remember like we work in teams usually and we're not an island right it needs to be a team decision because at the end of like in the end it's the team's effectiveness is more important than our own idealistic view of how an application should be structured because at the end of the day that is the technical detail whether file for control it is in this directory or that directory who cares really would I start every project like this now with this new approach probably not I would wait until I until I feel the pain of having a slow test read and then worry about okay how can I optimize this for example if I would put up a rails application for people to sign up for my birthday party that I will put on a free tier Heroku Dino scaffold the hell out of it and deploy it and call it they really it's really thinking about how much maintenance do you expect for this particular problem and how much benefit do you get from all these optimizations and we should remember to not just follow whatever has been done before us question things break things fix them again but break things and question things like take the leaning tower of Pisa for example it's a great attraction it is massive amount of people go there every year is that an indicator that we should build every tower like this now probably not and last but not least or more importantly know your tools like know why you follow certain rules but also know why and when to break them make it a conscious decision and on that note I think it's the time all right I will leave some room for some questions but first say thank you for listening to me
 Welcome everybody and thanks for coming. Today I'm going to talk a little bit about Rails  and how awesome it is. And I think part of why it is so awesome and why it is great is  its community and the huge ecosystem around it. And when I try to explain Rails to newcomers  to the industry or even colleagues, I think of Rails about this city map, this huge massive  city map. It's big and consists of different areas like over there is the active record  district and there is the initializer town and all these things. And in our day-to-day  work, it can feel kind of intimidating sometimes even. You're surrounded by all this functionality  and you don't know what's happening and you feel constrained sometimes even. And while  it's true that Rails has its certain ways and patterns how things should be done, this does  not mean that we cannot find or create our own little areas of freedom. And today I'm  going to talk about some of the areas we explored at AtheLite in order to help us work in a  more efficient way with Rails. So at the beginning I want to talk a little bit about the status  quo. Then we talk a little bit about breaking of some things followed by a quick wrap up  about general application architecture and then talking about some of the trade-offs.  So quick question to the round, who here knows status quo? A couple of hands? All right,  cool. So in contrast to Rails, status quo had a very simplistic approach. They always  only played with three chords. And while I'm not going to continue today to talk about  the band status quo, I thought coming here from London, I should at least take up a British  reference. So we're done with that. We continue with the actual status quo as in the current  state of affairs. Two years ago, DHAH mentioned at RailsConf that Rails for him meant that  he was going to use Rails for him. He still wants to be able to use Rails in order to  write web applications. And I think this is a really cool feature and idea behind it because  everything comes with Rails. Like batteries included, you can continue or you can start  writing web applications without the need of anything else. I'm more of a ultra-light  backpacking person myself. And while I'm not prepared for the zombie apocalypse, I always  have just enough gear with me for my current adventure. And this is reflected in the Pareto  principle, mostly known also as the 80-20 rule. So for example, 80% of your users use  only 20% of your product's functionality. Or in Rails' case, 80% of Rails applications  out there use only 20% of what Rails offers as features. And the background of this talk  has something to do with Omakasa Rails applications. And throughout several years, and having done  a couple of Rails projects, we felt the same pain over and over again. And that is that  Omakasa Rails applications lead to slow test suites. And this is a problem for us at Aethlite  because we are used to quick TDD cycles. We want to be able to write a little code, run  the test suite, to get feedback immediately. So this writing a little code and running  the test happens, I don't know, five to six times a minute. Sometimes if I'm typing sloppy  ten times. And I want this fast and quick feedback loop. Unfortunately, the bigger  your Rails code base grows, if you follow the regular Omakasa style, you lose out on  these things. And in a previous application where we felt the pain and it was the last  application we built in this way, this will be our reference application for throughout  this talk. And we ended the project with around about 4,800 tests and the full test suite  ran in about six minutes. And waiting six minutes until you know whether refactoring  worked or not is too long in my opinion. So I need my code base malleable in order to  make a quick decision. Yep, okay, this refactoring worked, continue, this buggy fixed, or it's  not fixed. I want to get this feedback really, really quick. So after having seen that throughout  a couple of projects, we went back to the drawing board, put on our thinking ads and  started to really dig into what is the actual issue. And this is me with my thinking add  on. And one hotspot we identified was the Rails Buddha process, which looks roughly like this  at the beginning, the Rails framework is loaded, then it will load all the dependencies from  the.jamp file and then it kicks off its initialized bang process, which loads all the code from  all configured Rails engine and so on and so forth. And during that process, something  like called eager loading or in the docs it's called or referred to as eager loading kicks  in. And eager loading requires or is responsible for requiring all code that is inside your  app directory, for example. And we started to modify our configuration a little bit.  And in a typical Rails application, you see in the application.rb, you see a line like  this require Rails all, require the whole world and continue. So we changed that a little  bit in order to only require the two Rails ties or the two frameworks we actually needed  for this particular application, which was only action controller and sprockets for asset  management. It turned out, which I learned after we did this change, that there's also  already support for this kind of configuration by Rails new. If you run Rails new with a  minus minus help flag, you get a list of all configuration options you can provide there.  And it actually allows you to skip a couple of plugins already and you see there skip  active record, skip action cable. If you don't need them, you can already tell Rails, hey,  don't generate the new project with these dependencies because I don't need them. And  I did run this command with a minus minus skip test because I usually use RSpec for  my testing. So what Rails new generates then is an application.rb that looks like this.  It doesn't require Rails all anymore, only Rails, which is the BAM minimum. And then  it actually tells you, hey, pick the frameworks you want, pick and choose. And it commended  out the test unit Rails type already for me because I was skipping that in the previous  step. So that's good and well. The next thing after requiring the initial plugins, there's  a line that will look like this. Bundler require Rails groups. In previous versions of Rails,  it looked a little bit different. It required the default and then the Rails nth. It has  been changed, I don't know in which version, but this tells Bundler to require all gems  from your gem file for your current environment. The problem with that is that this line will  add a linear load time to your boot process, which means the more gems you add, the more  time will be spent when you load up your application. And this is unfortunately a hard fact and  a hard truth we need to accept. At the moment, there is no way around that. So we continued.  There are a couple of more settings we started to tweak and one of them is active support.  Supports a bare flag in order to say, hey, don't load everything from active support,  but only the dependencies that are actually needed in order to boot up Rails. And then  there are two more settings, cache classes in order to keep a class cache. If a class  has been loaded, keep it loaded, and then we disable specifically the dependency loading.  And this is because at that time, we were big fans of a thing called screaming architecture,  which I will go to, which we will come back to at a later point at the presentation, but  as a quick heads up, screaming architecture, the idea behind that was that it should be  obvious for a new person on your team that to know by just looking at the directory structure  what kind of application is it. And if you take the Rails core base as an example, if  you generate a new Rails core base, you see app in config dblib public and I don't know  what domain is represented in this particular application. Just because I've seen Rails  before, I have an idea that seems like a Rails app. And if you drill down a little bit in  the app directory, there are even more sub-directories which do not really reveal the intent beyond  the application itself. So what we started doing is we moved everything that was in app  into the lib directory with a proper namespace. So in this example, there was a movie organization  application. So there you can see there's an edit index and show ERB template. There is a movie  model, movie repository, and also the movie controller. Like everything that used to be an app,  we moved to lib. We deleted the app directory altogether. In order to really make sure that  nothing will be eagerly or auto-loaded or looked up anywhere, there are a couple of more settings  for the application.rb. We made sure, okay, eager load false, don't try to load anything,  and we made sure don't even try to look up any path because we pretty much nulled them.  So we booted up our Rails application, no errors whatsoever. Trying to hit the index page, blew  up with an exception, uninitialized constant movies controller. So I mentioned before Rails  expects everything to be loaded up front. This is why this pre-loading, or eager loading happens  in first place. So now Rails can't find our application controller anymore. Luckily for us,  there is a thing called lazy loading to the rescue for us. And what we did there,  in the sector, you can see a call to active support, constantize. So at that time, Rails tries to  constantize a controller name based on the route definition we provided there. So because Ruby is  flexible and awesome, we monkey patched constantize. More specifically, this particular one line you  see there, we wrap the existing constantize and say, okay, if you try to constantize a controller,  actually require this controller as well. And this makes the trick, everything worked, we could  serve requests, everything worked fine. But it had an interesting effect on our code base because  now we needed to have explicit requires everywhere. And this was a constant decision within the team.  But it's also not very Rails-y, if you want to say so. What I like about it, it makes it obvious  how many dependencies my current file I'm looking at has. So I refer to this as a manual static  analysis. If you open up a file and it has 20 require statements, it gives you at least  a hint to, well, maybe this thing is doing too much. And then you can start thinking about  extracting things or splitting it up in different ways. Maybe it's okay, but at least you can start  having a conversation and think about, is this thing doing too many things? And with all these  settings now configured, and we were able to start a new green field project, which will be our  example one. So green field project, we had everything at our disposal. We could do whatever  we want because we were in complete control of what we want to do and how we want to approach it.  So we did all these configuration settings. And here's the existing or the previous application  again with the six minute runtime of the suite. I was not part of the project when we kicked it off.  I joined the project when there was already 5,500 specs that ran in 23 seconds. And while I don't  have a percentage here, I think this is a pretty big increase, decrease. Additionally to that,  with all the tweaks about how many files need to be loaded at the beginning,  loading or testing one controller in isolation used to take around 18 seconds until you knew,  okay, this controller works or it doesn't work. Now with the new approach and the new configuration  settings, we only needed to wait two seconds, which I think is still pretty good. One caveat here,  though, is we made also a choice to not use Active Record at all. We switched to the SQL gem  and a repository pattern in that way, which you don't need to discard Active Record for that.  It's mostly about that you can switch your implementations for your persistency  depending on the environment you're in. So when we were testing, we had an in memory  repository in order to help us gain more speed. In production, of course, we use a real repository  that actually connected to our Postgres database. But we could still have used Active Record for that.  It's just we switched to SQL. No particular reason. It was just the way we started working with it.  It's worth mentioning here, though, that this adds double burden on us. We needed to maintain  two separate versions for each repository. We needed to maintain a Postgres implementation  as well as an in memory implementation. But the benefits for that was that we gained a lot of  speed for our test suite. So it was worth for us investing into this double maintenance. The way  we did that was with shared RSpec examples that ran for both implementations. So we actually only  needed to write the test ones, but implemented twice, basically. The second example I have was  a Rails application we took over from a new client. And that started with 220 gems and a  liberal usage of Active Record. The application itself came with 2,600 tests out of which were  730 controller tests and the rest of them were not just plain or Ruby object tests. There were a bunch  of Cappie Bar tests as well as tests that integrated or verified some elastic search behavior as well  as verifying that the side gig job was kicked off. So it wasn't just controller and the regular Ruby  world. And the full suite ran in around 8.5 minutes. So we thought, okay, let's optimize it our way  in order to gain more speed. And running a single controller with the existing application  with the require Rails all way took around 14 seconds in that case. After we did all the  configuration changes and all the required statements because now we actually needed to  add all the required statements manually back into the code base which weren't there before  because it was a regular or MacArthur Rails application. We were only able to cut it down to  nine seconds. Still a little bit better but not the benefit we had before. And for a non-controller  spec it looked a little bit better. If we were running a single non-controller spec it took  around six and a half seconds. And we were able to cut this down at least to around half a second.  So there was more optimizations. There was a better optimization for us than with the controller test  but we weren't able to just apply it throughout the whole code base because manually adding all  required statements took a lot of effort. I did all the benchmarking for two controllers.  And at the end of it I had like my Git status showed me I had 150 piles changed. So because  it's a client project we can't build the client for refactoring that we wanted to do for refactoring  sake. So we would need to find a way to slide it in as we add more features. So we couldn't just  flip the switch and say yep now your application is faster and we have a better development  experience. But we could do some projections. So around eight and a half minutes was the previous  runtime for this week. And with the knowledge about like it's not just controller test and  non-controller test for us we projected numbers that we were able to cut it down to at least five  to six minutes which is still fairly long too long for my own taste but a little bit better than  before. And with all the changes and all the explicit requires I know that not many people  would agree with me. The main point I'm trying to make here is that launching a test suite  should not be a daunting task. You should you want to be able to run your test suite  multiple times a minute to get this fast feedback. And if you only take one thing away from this talk  is split up your spec halvers. Split your spec halvers in a spec halver that has  just that just loads our spec and maybe define some global test halvers that you want to be  using throughout your test suite and a separate rail spec halver that requires the existing  spec halver and also requires the rail environment. This way you have this is a really low hanging  fruit in order to at least optimize all your non-rails test. Like if you just test the class  that does not require anything from rails you get a huge speed benefit. Having said all of that  let's talk a little bit about the rails way. So what does this actually mean? In Rails's doctrine  we can read that there is a value convention over configuration. We made a very conscious decision  in our team that we favor expliciteness over implicitness. I like to read explicit and boring code.  Like magic code does not I'm not a you I don't really like it. That doesn't mean that I don't  like rails but it's just I favor explicit code over knowing that okay something will be loaded  and something will be done for me in the background. It's worth mentioning that conventions are  heuristics. They are not rules. You can break them and if you learn something new you might even  realize how much this convention we used to follow for three years. Maybe it's not a convention  anymore. We should use something else and it's not a bad thing. This comes the 80-20 rule applies  here again. Like all these conventions we find in rails work very very well for 80 percent of  rails apps out there but the remaining 20 percent might need something else and that is fine. It's  not bad. It doesn't mean that rails is not the right fit for them. It's just that rails still  offers enough flexibility in order to support the remaining 20 percent. You just need to maybe work  a little bit harder to reap these benefits. As I mentioned earlier I want to talk a little bit about  general architecture as well and while I'm not I don't want to bore you with 90s 70s material  about what software architectures should look like there are a couple more recent architectures out  there. First one is clean architecture which defines some entities and the core of the application  that will be orchestrated by so-called use case implementations and then these use cases will be  exposed through controllers in our rails application for example through the web.  In a similar vein there is a thing called hexagonal architecture coined by  Alistair Coburn which goes in a similar direction like at the core of your application is your  actual application and you provide different adapters for different clients. Like you can  have a GUI adapter in order to connect the QT GUI to it or an HTTP adapter to connect maybe a browser  to it like think of rails here and these adapters do not only work for like inbound connections  also for outbound connections like you have an adapter in order to connect to your persistence  layer Postgres on MySQL or you have an adapter for the Arrest adapter that connects to Salesforce for  example and last but not least in Martin Fowler's book Patterns of Enterprise Application Architectures  this is a thing called service layer same idea again you have a user interface that  connects to service layer which guards your domain model which then it itself has access to this  it's called data source layer here it's your database at the end of the day and when we look at  these three architectures they look very very similar they all have the same idea behind them and  it's not something completely new even because if we take the service layer as an example and we  focus on one particular area and we zoom in on that it's pretty much a layered architecture again  like nothing new to learn here we're back in the future back in the 90s it's fine like new words  for old concepts like isolate your application from the outside world that's pretty much it  it's important to remember though that Rails is not your application and this is why I was emphasizing  on the architectures a little bit and I want to show you an example of a pictures e-commerce  platform here again in e-commerce platform we might have a user class that if user logs in we  have a customer class representing this particular user and the user can have a basket a basket can  consist of a couple of products and each product has a category and while we're checking out we need  to select a delivery method as well as a payment method and at the end of the payment process we  get an invoice at a PDF download for example and this is our core application like we don't see a  controller there we don't see a session hash or any gnarly rails of all web details like this is  the core of our application and this goes in the direction of the manager of design a book I highly  recommend to read and in order to tie that back to the architecture we saw before let's have a look  how that applies to our res application how that would apply to our res application so if we start  from top to bottom our user interface is usually a browser and this browser connects through HTTP  to our res application or our checkout controller and the checkout controller then translates this  web request to a thing called our checkout which is our use case implementation in this instance  and the checkout process the checkout use case then mediates between several domain model objects  which ideally should just be plain old Ruby objects that define some behavior and then at the end  it will create an order which is or which acts as our gateway to our database and this is pretty  much it like top down one request comes in and you might have something new in the database  how does that tie back to Ray as this MVC idea  MVC has become this more of an abstract concept than a design pattern or anything like that  and we've seen that a couple years back in the reds community even  where there was talk about oh yeah we should pay for fat models over two skinny controllers  until we realized okay these models get really really hard to test so we went to the other extreme  and said okay let's keep the models really thin and put all the behavior in the controllers which  made it really hard to test the controllers and it's not enough to only think or talk about these  three buckets because I see this as a balloon that you just squeeze on one end you're not magically  removing air or responsibilities from from things and you're just squeezing it here with  and it will expand the balloon on the other side so it's important to not think about  or focus on skin or fat anything we should focus on a healthy everything because spreading the  responsibilities into small chunks will help us make create an easier to understand code base  like if we only worry about classes that have five to ten lines it's really really it should be  easier to understand than following a 50 method long method for example  and when I was preparing this talk I had a chat with a friend over coffee and I was going through  the topics and areas I want to discuss with him and he was asking me okay so we did this type of  rails style for a couple of years now and he was asking would you do it again would you still do all  your applications like this and I was thinking a moment and I thought well all the pre loading  and loading optimizations yes for sure but the idea with the screaming architecture like removing  the app directory and moving everything into lib probably not I think this is utterly overrated  because if we think back to the default directory structure I don't necessarily need to know that  this is a movie organization database or e-commerce platform for flowers or something like that  it's okay to start with okay it's a res application if I want to know more I probably look into  lib or if there's a source directory there's source and then I have my nice namespace responsibilities  laid out there and with the separation of concerns or like separating your application from  everything that is rail specific I actually like the idea to have these physically separated now  like my core application is in lip nothing rails specific should leak into lib everything rails and  web related should stay in app like this makes it not trivial but at least easier to upgrade rails  as well because we don't need to worry about anything that is in lip maybe we wrap some active  job dependencies inside lip but this is really about it lip is our application nothing of rails  should leak in there so everything comes with a tradeoff right I'm not trying to sell you the  promised land here like it's not that magically all of a sudden we will end up writing more and  more awesome rails apps there are tradeoffs and there are hard tradeoffs you make for example  explicit requires this is something that is not necessarily used in a typical rails application  and if you take back the city example or the city metaphor at the beginning like we put up some big  construction sites and just cut through a couple of blocks not without thinking but  it we need to be aware that it comes with the cost because we need to remember like we work in teams  usually and we're not an island right it needs to be a team decision because at the end of like  in the end it's the team's effectiveness is more important than our own idealistic view of how an  application should be structured because at the end of the day that is the technical detail whether  file for control it is in this directory or that directory who cares really  would I start every project like this now with this new approach probably not I would wait until I  until I feel the pain of having a slow test read and then worry about okay how can I optimize this  for example if I would put up a rails application for people to sign up for my birthday party that  I will put on a free tier Heroku Dino scaffold the hell out of it and deploy it and call it  they really it's really thinking about how much maintenance do you expect for this particular  problem and how much benefit do you get from all these optimizations  and we should remember to not just follow whatever has been done before us  question things break things fix them again but break things and question things like take the  leaning tower of Pisa for example it's a great attraction it is massive amount of people go  there every year is that an indicator that we should build every tower like this now probably not  and last but not least or more importantly know your tools like know why you follow certain rules  but also know why and when to break them make it a conscious decision  and on that note I think it's the time all right I will leave  some room for some questions but first say thank you for listening to me
Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_0_395.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_396_1739.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_1740_1799.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_1800_2951.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_2952_3059.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_3060_3191.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_3192_4295.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_4296_4499.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_4500_6575.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_6576_6887.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_6888_7487.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_7488_8819.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_8820_11999.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_12000_12863.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_12864_13427.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_13428_15527.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_15528_16079.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_16080_16223.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_16224_18611.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_18612_18899.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_18900_19751.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_19752_24287.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_24288_27419.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_27420_27827.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_27828_28739.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_28740_28871.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_28872_30911.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_30912_31451.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_31452_37931.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_37932_39179.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_39180_41015.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_41016_42947.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_42948_43079.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_43080_43595.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_43596_44747.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_44748_47315.png Breaking_Bad_-_What_Happens_When_You_Defy_Conventions_2_47316_47350.png