Modularization
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Nilsson, Elin
Modularizing Android codebases is all the hype these days. It can be as simple as moving a few files, and maybe creating a new build config or two along the way, or it can be much more gruelling. At Spotify, it has meant clearing out years and years of tech debt, re-writing APIs, rethinking the architecture and structure of the application and features that goes into it, wrestling with build tools, and then some. This talk will walk you through the Spotify modularization journey: why we got started, how we went about it, what we have learned through the process, and also why you should be thinking about modilarizing your code, if you haven't already started.
 Hello, everyone. Thanks for coming. My name is Aelyn. I work as an Android developer at Spotify and one of our platform and infrastructure teams. And how many of you have worked with modernization? It's quite a few. Cool. So yeah, that's what I'm going to be talking about today. And given that you've all worked with it, you probably know the answer to this question. How hard can it be? Keep it very hard. So that's like the TLDR of this talk. It's very hard. It can be very hard. But the longer story, it's always more complicated than that. So modernization is a very hot topic. These are some links to some talks and blog posts that have been done on modernization over the last year. There's also been some talks actually during DroidCon now. We had Nikita Mboris who talked earlier today about modernization. Ben talked about doing an Instant App. Vajtek yesterday was talking about Instant Apps and App Onlux and stuff like that. And this, I think, is absolutely great because they're covering stuff that I will not be talking about in this talk today. I will not be talking about Instant Apps or dynamic delivery. I will not be talking about tooling or data or numbers. I will not necessarily be talking about code, actually. Because modernization isn't about code. It's just about moving files when it gets down to it. But what I will talk to you about is the Spotify modernization journey. And I think we're going to talk about why we got started at Spotify with modernization and how we did it, what we've done, and what we've learned. And I guess what I'm hoping that you can get out of my talk rather than all the other talks that already exist of modernization is talking about modernization more at scale. But yeah, to get into why we started modernizing at Spotify, to know that you need to know some history about Android development and mobile development at Spotify. So mobile development at Spotify started around 2009. And since then, it's been growing quite a lot. And Spotify has grown in many different ways. The amount of teams have grown. The amount of developers in those teams have grown. And the amounts of features in the app has obviously grown over the years. And what we all know is if you have more features, you will have more code. If you have more code, it becomes very slow. And this was the main reason why we got started with modernization. And I say reason. It's more that I think build times for us was the reason why we couldn't not do modernization. Because at our scale, our build times were grinding to a halt. And I think had we not started with modernization when we did, hours of build times, I don't know if that basically we wouldn't have been able to work. And that's bad. So we decided to, OK, let's do this. We need to modernize. And we got this task force team that started out with modernization. And it was four or five people who just sat down to figure out, OK, how the hell do we get through this? Because we need to, in order to not die of boredom and slow build times. And then how they got started initially was they, yeah, we had this core team. We sat down. And yeah, also to preface, we did actually not start with just one module. We had 40 modules when we started out. And that was, we already had modules. So some of our basic UI stuff was already modernized from scratch. We had some testing frameworks and stuff like that. And partner SDKs and things that were already in modules. This is about our main monolith module. So yeah, we had this team. And they sat down. And they created this goal, which was to get one feature outside of the main app. And this feature was a quite small, seemingly contained feature that no one had touched in years. It was sort of sitting in a corner somewhere. And we were, that was the goal. We're going to get this outside of the main app. And as a sub goal, we also had, we want to be able to run the unit tests for this feature in under five seconds. Then they got started with some upfront work. Some very basic tooling. So this was like, we created a script to set up a module and create build gradle files and add it to the settings gradle file. And just like, you give it a name, and then you have a module. Just basic things like that. And then we also aligned on the basic structure. And this can be a picture. Wow, that's really shit contrast. Anyways, this can be, if you have many different modules, they might look like this. And this is, everyone likes bubbles. The structure we went on is our main app. So app is the main module. I'm going to refer to it as app, but it's the monolith module. But we basically saw that as the big thing that we wanted to pull stuff out of. And then under that, we had features, we had common code, and we had libraries. And now, it doesn't seem like you can see that those down there are still bubbles. And I guess my point with this is we didn't make just four modules and have everything in four modules, because that doesn't help anyone. What we did instead is the features, common libs, those are more categories for the modules that we did end up pulling up. So our feature modules, that's stuff like end user features. So search, radio, your library, stuff like that. Common code is stuff that is common code that we share in Spotify, but it's still Spotify stuff. So we have some UI components sharing. I have radio here again, because radio for us, we have a radio feature. But then you can start a radio from anywhere in the app. So radio is a feature, but there's also common code about radio, which is used in many different ways. So that's structured in two ways. And then the libraries, those are things like very basic stuff, like HTTP, like logging, like something that, and yeah, the reason why we're separating common and libs is common is stuff that is common to Spotify. Libs could be open sourced, technically. It could be any third party thing. It's not app specific. So if you want to build more apps in the future, we can still share code through the libraries, but not through the common code. And I mean, we do have another app called Spotify for Artist, and they use some of the code. So we do have shared code in the libraries, just not in the common. So there's also a set of common modules for that app. But whatever, that's the main structure that we set up on. And having done that, we started pulling out everything that we needed to make this one feature work that we wanted to pull out of the main app. We also sent an email saying, hey, we're going to moderate. We're probably going to break stuff. We're sorry. We're going to buy a cake if we break master. Yeah, it's just a nice thing to do. So then we said, OK, we got the team. We got the time. We have the PMs on our side, and we're allowed to do this. Let's try it. And pulling stuff out initially, it doesn't always go that well. You move some files, and you're like, oh, look, it's all nice. And there's no squeak-lea-lands anywhere. And then you try to compile or run in it. And yeah, it's a struggle. So then we tried it again. Then we tried it again. And then after a while, it did actually compile. And we did manage to get that one feature out. And you have the sort of side goal that we had on running Unitess in less than five seconds. That was sort of like, we'll pull it out and see if it's possible. And it was. We didn't have enough to do anything. So yeah, if anyone were to use that feature and do more development on it, they could use TDD and not have to wait for three minutes for Unitess to run. So through this trial and error in the beginning phase, we came up with this process. And I like talking about process through pseudocode algorithms. So I'm going to go with that. So the first thing you want to do is you want to analyze dependencies of anything that you want to move. We have the scale for dependencies, from frictionless to massive friction. Where frictionless, that's classes that you can just move them. They have no dependencies to anything. If you move them up, they still work. And then we have a scale of slight friction to more friction where you have a class which depends on a class that's frictionless to move. Or you depend on a class that depends on another class and that in turn depends on a class that's frictionless to move. So there's sort of an infinite scale of things between more friction and massive friction. But the massive friction, that's like the main source of any spaghetti code in the world. In our case, one of those is main activity, which is, let's start it out. I'm sure it was very small at one point. It's not. It basically knows about everything that's going on in the entire app. There's basically no way in hell that would ever not live in the main app module. So yeah, that's the scale. So when you've analyzed your dependencies and you know, OK, this is the dependencies, then you go a few number of ways. If it has no dependencies, you just move it. If it asks, yeah, remember tests. We have a lot of abandoned tests where we've moved out code. And then we forgot about the test because they never show up when you analyze the dependencies. And then they get sad. I mean, they're still in the app module, so they still run. But it's nice to have code in the same place. Yeah, if you have simple dependencies, you can just break those dependencies. And simple dependencies here could be something like you're depending on a class, which is like a utility class. And I mean, it's nice with don't repeat yourself and having multiple codes over the same things. But you could probably inline that. And it wouldn't be that big of a deal. Other simple dependencies could be you might have a dependency on something because you use it as a parameter. But maybe you just send that through to somewhere else, which actually needs the thing. And then you could just maybe switch that parameter to something else or just use something else or do something else. Maybe the code is old and you can just use a new API or something. So then you break the dependency and then you move on with your life. If you get into the complex dependencies, then you need to spend a lot more time. So then you basically need to spend a lot more time doing the refactoring. Or you could try doing dependency inversion. And I'm going to talk a little bit more about that later. And if you're so successful with breaking your complex dependencies, that's great. But probably you won't be successful. And one thing we did was we introduced this just very simple annotation called Moderization To Do, where we just put that in the code base where we ran into a wall and we couldn't do anything else. We added a Moderization To Do and then the reason for it. And then that makes it just very easy in the code base to just keep track of where we need to do stuff. And then revisit that. Like if you must align, maybe we've already modernized the bigger dependency that we couldn't move because of. But I mean, it's sad. You spend a lot of time. So you might stop quite early in the corner before moving on with your life. Because the main reason and the main lesson that we learned really early on in the process is modernization takes time. And you need to be patient with modernization. But like I said, we managed to get that first feature out. We managed to get more and more features out. We pulled out more and more code in order to just make the app as small as possible. We did have a time in the beginning where it was just like anything just like, let's get it out here. Just standing with shovels. And yeah, we were just keep doing this and doing this. And we were keeping metrics of all of this. And we were just seeing like, we're moving so much code, but it's not getting smaller. What's happening? All right, we're still shipping features. That's right. There's still adding stuff. So it's, you know, it's tough. So then we're like, hey, can't you just add your code outside of the main app instead? Because then the main app doesn't get bigger. But they couldn't because we didn't have, we hadn't modernized this one API yet. Or like this one thing yet, because that's still in the main app. So unless you move that, we can't do anything at all. And that's also started out as thinking, so that's not really the case. Maybe you can't move your code immediately or create it in another place immediately. But you can write it as if you had it in a module. Because if you write your code as if it's in a module, if you then put it in a module, it will be a lot easier. So this leads to best practices. And some of the best practices that we developed and talked about throughout this process, I'm going to share with you now. So one of them is sort of, don't repeat yourself from moving dead code and increasing reuse. And my main point here is this kind of touching on it. And it's you're probably doing too much in your code. And you can probably do it in a more organized way if you think about it as, what things am I actually doing? Am I doing, I'm probably talking to a backend somewhere to get some data or doing something like that. And that's often coded that is repeated. And you do it in maybe different ways, in different features, or in different places of your code. But what you could do instead is just to have one API for this is how we do HTTP calls. And then that implementation can be the same and shared. And that logic doesn't have to live everywhere in the code. Another one, which I touched on, is avoid parameter dependencies. So an example for us is we have a feature flagging system that allows us to do A-B testing and graduate rollouts and stuff like that. And what we have to check that is we have this flags object. And we do a lot of A-B testing, which means that flags exist basically everywhere. And existing classes that don't really care about flags or they're not using them, but it needs to be there because somewhere someone's using them. And this is what I was talking about, that we're using these flag objects to just check if something should be on and off. And instead of sending flags all over the place, we can just send this as a Boolean. And then we don't have dependencies anymore. Also makes the code a lot more readable because rather than seeing flags absolutely everywhere, you have Booleans going like, should this button be green or not? That's just easier to read. And that ties into dependency injection as well. And having dependency injection means that rather than passing around flags because somewhere needs it, if you're using dependency injection, you can just inject flags where you actually need the flag and where you want to do that check and not just have it all over the code base. Structure your code. And I realized, I rewrote this. I had this as change the structure of your files and whatever. And then I changed it. And I realized that this is how should you structure your code? Basically in any way, having structure is better than not having a structure. And having a structure that's similar across features is good. And that doesn't really matter if you're structuring it. You have UI in one folder and models in one folder and helpers in one folder. Doesn't really matter if you have that or something else. But if it is consistent across features, it helps out along the way. It ties into the stuff I was talking about with having clear APIs for the kind of stuff that you're doing and knowing what you're doing in your code. Because if you structure it in UI here and then HTTP here and then this here, then you might see that, oh, we're doing a lot of custom HTTP logic where we could just use this more simpler interface over here instead and not have to do all of this ourselves. Avoid using utility classes and static methods. And this ties in again to the parameter dependencies because at least at Spotify, we had a lot of utility classes and a lot of methods. And what happened with those is because they're static, obviously, and then you just send in all of the parameters, just all of them. Then maybe something changed in the utility. Maybe someone introduced an A-B test and something needed to change. So then all of a sudden, everywhere needs flags all of a sudden. And again, just having objects instead, like helper objects that inject these things that might not be required for everything everywhere to use. But this helper class needs to know about flags. Then you can just create an object, inject the things you do need, and have the parameters be just the things that actually is needed from the thing that is using the helper. And then solid principles. And solid principles, I hope most of you know about them. I'm going to run through them. This is our TLDR version of them. Don't do a lot of things in one class. Design the API so that a class can be used and extended without being modified. Don't let implementations of the same interface behave differently. Don't design big interfaces. And depend on abstractions, not on implementations. And this last one, which is dependency inversion, this has helped us immensely with modernization. And yeah, basically what it is is, instead of having your class directly depend on or use method of another class somewhere, you put an interface in between. And this is very helpful in those examples where you have massive friction to be able to move whatever you're using. So for instance, if you're, in our case, if we have anything that relies or depends on the main app or the main activity, instead of trying to move the main activity, we just make our code dependent on one interface, which could be like navigate to home. You often, it can be navigation, but it could be anything. And then you just have a simple interface, which has no dependencies whatsoever. And then you depend on that. And then if you're using dependency injection, you can bind the actual implementation of it, which might be something like main activity or just a class that uses main activity. And then you bind that. So you're injecting the interface. You have no idea about main activity, but under the hood, it's still calling main activity. So what have we actually done up until this point? So modernization for us started just over a year ago. And since then, we've moved a lot. So I don't know if I mentioned metrics, but we were collecting metrics all throughout our modernization phase. And the reason the graph is smooth at one point is we don't have a lot of data points for that, because we didn't need to, because we weren't changing anything. But I think you can basically see where we started modernization, which was around, I don't know if this is visible at all, but around beginning of 2017. And this is the period I was talking about where you can see. And at that point, the app, the main app module around that stagnant point there, was around 600,000 lines of code. And yeah, you can see that stagnating out, but you can also see the amount of code in the app growing like crazy. But the main app is still stagnant. And that was the bit where we were just shoveling out water of the sinking ship and then feature developer, sort of just pouring in more and more and more. And then something that's interesting with this is at the end of 2017, you can see the main app module starting to quite slope and actually decrease properly for the first times. And that was basically when we in the modernization team stopped working on modernization. So this was when we got to a point where we've modernized as much things as possible. It's now on you, the feature developers, to move out your stuff. And at that point, we've gotten to a place where they could do that. So we got people to do it. And now it's down to, it's actually below 300,000 lines of code right now. It's almost 290,000 lines of code in our monolith module still, which is basically back to when we were three years ago in 2015. So that's the journey, that's the graph to show in our journey. But it's still very painful. I'm going to show you one more graph. And in this graph, the black line at the bottom, that's the red line here. So that's the total lines of app in our code base. Does anyone have a guess of what this green line is? This is our files. This green line is the lines of code of our files that we have. And we have 25 million lines of our file code. And this is, yeah, we have around 400 modules at this point, slightly over 400 and growing. We have over 14,000 r-files. And the main reason for this is if you have a module, you have an r-file. If you have another module, you have an r-file. If you depend on that module, you have three r-files. Because this module will have its r-file, and that will have its r-file. But this will have an r-file of that module as well. And this is something that Greta a lot aware of, and they're trying to work on and improve for us. It's not a problem in Basel or Buck. But we're not using those, so it's painful. So yeah, let's talk about build times. And now there was a talk earlier today, and they went into a lot more numbers about build times with modernization. But basically, if you have a smaller module, it's faster to build and compile incrementally. But the bottleneck in your build times will always be the size of your main app module. And now you have stuff like Basel and Buck, which are other build systems. Basically, it's built by Google, Buck is built, and maintained by Facebook. And Basel and Buck are a lot better at handling builds when it comes to a modernized code base, given that you don't have big modules. So I had a colleague actually who tried out Buck and using OK Buck at Spotify, and the build times for the modules became stupidly slow, like two, three seconds. But then the main app was like 15 minutes. So we have a ways to go. So yeah, I'm going to move on to what's the biggest lessons and the biggest takeaways that we've learned throughout this over a year-long process. And I kind of like this quote. It's hard to see the forest for all the trees. It kind of relates to there's this idea called the broken window theorem. You may have heard of it, which is if you have an old building and you have one, if you have no broken windows, you will not have more broken windows. But if you have one broken window, all of a sudden you'll have like 20 broken windows, because there's already one, and they're not fixing it, so we can just break everything else. And why I like this and why I bring it up, talking about monetization, is that it sort of goes hand in hand with having a big module and all the code at one place. You can't really see all the tech that you have and all the spaghetti code you have. And you can't really tell if you're introducing code that increases the build time by 10 seconds. If you have a build time that's already minutes long, maybe you won't even notice that. But if you're introducing that in a module where the build time is 10 seconds, and then you're adding 10 seconds, then all of a sudden it's double, like, that's 200% or 100%. So if you're in that place, it's easier to see when problems pop up. And that's how I see monetization. I see monetization as a tool to get your code base and your code to be healthier. And to make it easier to know what dependencies you have or what the code quality is. It's a chance to tackle your tech depth. Like I was talking about with the best practices, because everything is small and encapsulated, you can't really have tech depth in the sense of complex dependencies and whatever, because you can't have those because you're in modules. Or you can, but it's easier not to have them. And I also see monetization as a chance to get to know your code base. And again, this is what I was talking about with getting to know what kind of stuff you actually do in your code base and knowing what utilities you use and knowing what things are in your feature codes and whether or not that stuff is specific to the feature itself or the piece of code that is doing stuff or if it's stuff like technically, if there was a nice API somewhere, I would use that. But there's not, so I'm going to build it myself. And then that happens in 10 different places in the code base, and then all of a sudden you have a lot more code than you need to have. If that is in modules, it's easier to spot those. And it's easier to think about nicer APIs. And there's this Yelp article or blog post that they posted very recently about how they did monetization at Yelp. And I thought it was interesting, especially this first sentence, single modules are easier to deal with. There are no complex dependency graphs to account for. And I thought it was interesting, and I just did revise it, because I think single modules appear to be easier to deal with. Because they don't have a complex dependency graph expressed in build Gradle files, but the dependencies are still there. The dependencies are still there between the different codes. But they're hidden because everything is in this main module. That ties into this module thing. If you have things in modules, you know about the complexity of your code base in a completely different way. Let's say some last wrap-up words of wisdom if you're going to start doing monetization, or if you're thinking about it, or if you're doing it already. Investing in some upfront tooling, like having a script for creating modules, doing dependency versioning, like having dependency versions for external third-party libraries in one central place and not have that copy-pasted in build Gradle files all over the code base, and then those getting out of sync, have it in one place. Also, static analysis for realizing when you've forgotten resources files, or stuff like that. Oh, I accidentally left things behind in the main app. And remember to move your tests. And don't change anything as you go. And this is like, you go into your code base, and you see a file, and you want to move it, and you look at the code, and it's like, oh, I can refactor this a little bit. And it's kind of old code. Don't do it. If you want to get monetization done, focus on the task. Do not clean up as you go. Don't do anything. Don't even change the packet structure if you don't want to. Just move it out, get it out of there, focus on the task at hand, add to do's, add it to the backlog to go back and revisit it. But do not do it as you go. You will lose yourself, and you'll never end up where you want to be. Inventory your app. Again, this ties back to just knowing what kind of stuff should probably be in modules, rather than scattered around in different feature codes. Can just help you to reduce the size of the modules when you already have them. And then also collect metrics, because graphs are fun. And then maybe you can go to conferences and do cool presentations with cool graphs and stuff. So I'm going to circle back to this initial question of monetization. How hard can it be? Can be very difficult. And I'm seeing it's a software development. How hard can that be? Can also be very difficult. You can make it easy, or you can make it hard. If you follow best practices, and you structure your coding clean ways, and you have nice APIs, and all the good stuff, it can be easy. It can be a lovely experience to do software development. Or you can accidentally turn everything into a huge ball of spaghetti, and then your life is terrible. I'm just going to leave you with this. All I'm satisfying is the channel name of our modernization Slack channel, where we do modernization support. And it's basically doing modernization. It's stressful. It's tedious at times. It's very painful. But in ways, it's oddly satisfying. You move out stuff, and then you hit a dent end, and then you cry in a corner for a bit. But then you revisit it a few weeks later when someone's moved something else. And then it's like, oh, look, I can move this now. And I don't have to worry about it. After the while, you don't refactor stuff as you're going along. But then you do. You come back to it. And then you have all this nice, clean code. And it's just a few classes. And you can go through them, and everything is nice and readable, and you can clean it. And it's just lovely. Thank you for listening. [? APPLAUSE?] And I don't know if we have time for questions. Well, yeah, we have time for questions. So please, everyone, use your last chance. Wow, tons of questions. Yes, please come here and walk to the mic. And I'll get the second one for you. Or you come over there as well. And everyone else, if you want to leave, please try to be mindful of the others that still want to listen. So be silent. Thank you. Hi. My name is Julio. I work at SoundCloud. And first of all, I want to thank you very much for the talk, which I think was one of the best talk of the conference up to this point, the last talk also. So I'm very curious about, after 400 modules you said, I'm curious about two things. How is your team organized? Like, how many developers you have in general? And how many are working in modularization specifically? Or if everyone is working in modularization? And then number two is, what do you think, or what's your advice for the base module? What should be there? OK, well, the first one. We have over 100 monthly active contributors to our Android code base. And we're organized in over 60 teams in five different cities around the world. We have New York, Boston, London, Stockholm, and Gothenburg. So it's a lot of people. And the modularization team, when we were starting out, it started with three or four or five. And then it grew. So we were eight or nine at the initial points. But then as we've gone, it's more like teams are modernizing what they own. Like, they're modernizing their own code. As for the other one with the base module, actually, like I said, I won't be talking about Instant Apps because we don't have Instant Apps. We don't have a base module. So I have no idea. I would actually like to know myself. Yeah. Thank you. Hi. Thanks for the awesome talk. One concern I also have in my project where I have about 20 Gradle modules is the R.java files as well. So one small note, I've watched a talk on Google Iow this year by the Android Build System team. And they announced that they are working in it. So hopefully one day that won't be any such an issue like now it is. And I have a question regarding the domain layer or the model, like the plain model objects. For instance, you have a song model. Do you share it among the other modules? Is it separate? Is it moved to some kind of a lib model or something like that module? Or do you have your own models per every module? And you have to reassign all the data when passing this object through other modules. Yeah. So yeah, we have actually great contact with the great old team. So we actually know about it as well. And we are hoping that they will do it soon. As for the other question, I'm so tired already of Gradle. It was about the modules. Oh, yeah. Yeah, models. Yeah, sorry. Should go home soon. Yeah, with the models, it depends. We have some modules that are definitely shared between different features and those we have in some libraries, or in common shared code. But there's also cases where a feature actually has their own models because they're not really sharing it. So in that case, it goes into that module. OK, thanks. Hello, I'm Mario from N26. My question was about static analysis. You mentioned it. Are you using any libraries out there? Or what kind of tool are you using for static analysis? To kind of check that while moderating something, you just did something, for example. Yeah, so we're using lint, PMD, find bugs, check style. Those are the main ones. We also have one that we've built ourselves, which is called lift lint, which is lint only on the stuff that you've changed. And the main thing there is lint will find all the unused resources and complain about that. And it does complain a lot. And then one thing that I found, which I think it was find bugs, which so the thing when you moderate stuff, it can build and run completely fine in some cases. But then at some point, it just breaks down. And you have no idea why. So we have this thing where a lot of just really strange problems that popped up when we moved files came up through our CI. And a lot of that was actually caught by find bugs that said that, oh, something can't compile. And it made absolutely no sense, because the file it was complaining about actually existed, but not something that was dependent on that. So that helped in knowing that something's not right here. But it didn't really help with pointing out exactly what that was. I think if there was more tooling for figuring these things, like we created a document with the most common. If you get this thing, it actually means this. So we have this translator for all the errors that we got through built errors. So tooling could be very helpful if it existed. And the last question. Hi. So yeah, question. First, in your features model, do you have also the activity and the Android stuff? So you manage with the Android to do the dependency injection in the modules. So a lot of our features are actually fragment based. So the fragments we can move out. But we can have activities outside. But then with the activity components, that gets a little bit hairy. And yeah. OK. Because I solved it with Coint. But Ricardo from SoundCloud told me that the Android apparently does it. One more question. Resources, like the strings, do you have them in the main module? Because I try to put the strings in their own module. But the localization team kind of wanted to kill me. Because a million different strings file. Yeah. Yeah, so we do have one main strings module. And that has translations. And we set up our translation scripts region smartling. And we've set all of that up for that specific module. But we also have some strings that are actually in the modules themselves. If they have strings that are, you know, they only exist for their own use. Because are the general strings files, that's like it's a lot of strings. That file is very, very long. If you can avoid being in it, it's nice. So then people have their own strings and their own translations within their own modules. So you wire the translation system to every single module. Yeah, we have a script that helps out with it. Perfect. Thank you. OK. So thank you. I know there are still some questions left. And I hope, Ellen, you have a couple of minutes for private questions afterwards. Yes, I also have stickers. And stickers. Wow, OK. Thank you, everyone, for saying until the end. Thank you.
 Hello, everyone.  Thanks for coming.  My name is Aelyn.  I work as an Android developer at Spotify and one  of our platform and infrastructure teams.  And how many of you have worked with modernization?  It's quite a few.  Cool.  So yeah, that's what I'm going to be talking about today.  And given that you've all worked with it,  you probably know the answer to this question.  How hard can it be?  Keep it very hard.  So that's like the TLDR of this talk.  It's very hard.  It can be very hard.  But the longer story, it's always more complicated than that.  So modernization is a very hot topic.  These are some links to some talks and blog posts  that have been done on modernization over the last year.  There's also been some talks actually during DroidCon now.  We had Nikita Mboris who talked earlier today  about modernization.  Ben talked about doing an Instant App.  Vajtek yesterday was talking about Instant Apps and App  Onlux and stuff like that.  And this, I think, is absolutely great  because they're covering stuff that I will not  be talking about in this talk today.  I will not be talking about Instant Apps or dynamic delivery.  I will not be talking about tooling or data or numbers.  I will not necessarily be talking about code, actually.  Because modernization isn't about code.  It's just about moving files when it gets down to it.  But what I will talk to you about is the Spotify  modernization journey.  And I think we're going to talk about why we got started  at Spotify with modernization and how we did it, what we've  done, and what we've learned.  And I guess what I'm hoping that you can get out of my talk  rather than all the other talks that already exist  of modernization is talking about modernization  more at scale.  But yeah, to get into why we started modernizing at Spotify,  to know that you need to know some history about Android  development and mobile development at Spotify.  So mobile development at Spotify started around 2009.  And since then, it's been growing quite a lot.  And Spotify has grown in many different ways.  The amount of teams have grown.  The amount of developers in those teams have grown.  And the amounts of features in the app  has obviously grown over the years.  And what we all know is if you have more features,  you will have more code.  If you have more code, it becomes very slow.  And this was the main reason why we got started  with modernization.  And I say reason.  It's more that I think build times for us  was the reason why we couldn't not do modernization.  Because at our scale, our build times were grinding to a halt.  And I think had we not started with modernization  when we did, hours of build times,  I don't know if that basically we wouldn't  have been able to work.  And that's bad.  So we decided to, OK, let's do this.  We need to modernize.  And we got this task force team that  started out with modernization.  And it was four or five people who just sat down  to figure out, OK, how the hell do we get through this?  Because we need to, in order to not die of boredom  and slow build times.  And then how they got started initially was they,  yeah, we had this core team.  We sat down.  And yeah, also to preface, we did actually not  start with just one module.  We had 40 modules when we started out.  And that was, we already had modules.  So some of our basic UI stuff was already  modernized from scratch.  We had some testing frameworks and stuff like that.  And partner SDKs and things that were already in modules.  This is about our main monolith module.  So yeah, we had this team.  And they sat down.  And they created this goal, which was to get one feature  outside of the main app.  And this feature was a quite small, seemingly contained  feature that no one had touched in years.  It was sort of sitting in a corner somewhere.  And we were, that was the goal.  We're going to get this outside of the main app.  And as a sub goal, we also had, we  want to be able to run the unit tests for this feature  in under five seconds.  Then they got started with some upfront work.  Some very basic tooling.  So this was like, we created a script to set up a module  and create build gradle files and add it to the settings  gradle file.  And just like, you give it a name,  and then you have a module.  Just basic things like that.  And then we also aligned on the basic structure.  And this can be a picture.  Wow, that's really shit contrast.  Anyways, this can be, if you have many different modules,  they might look like this.  And this is, everyone likes bubbles.  The structure we went on is our main app.  So app is the main module.  I'm going to refer to it as app, but it's the monolith module.  But we basically saw that as the big thing  that we wanted to pull stuff out of.  And then under that, we had features, we had common code,  and we had libraries.  And now, it doesn't seem like you can see that those down there  are still bubbles.  And I guess my point with this is we didn't make just four  modules and have everything in four modules,  because that doesn't help anyone.  What we did instead is the features, common libs,  those are more categories for the modules  that we did end up pulling up.  So our feature modules, that's stuff like end user features.  So search, radio, your library, stuff like that.  Common code is stuff that is common code  that we share in Spotify, but it's still Spotify stuff.  So we have some UI components sharing.  I have radio here again, because radio for us,  we have a radio feature.  But then you can start a radio from anywhere in the app.  So radio is a feature, but there's also  common code about radio, which is used in many different ways.  So that's structured in two ways.  And then the libraries, those are things like very basic stuff,  like HTTP, like logging, like something that,  and yeah, the reason why we're separating common and libs  is common is stuff that is common to Spotify.  Libs could be open sourced, technically.  It could be any third party thing.  It's not app specific.  So if you want to build more apps in the future,  we can still share code through the libraries,  but not through the common code.  And I mean, we do have another app called Spotify for Artist,  and they use some of the code.  So we do have shared code in the libraries,  just not in the common.  So there's also a set of common modules for that app.  But whatever, that's the main structure that we set up on.  And having done that, we started pulling out everything  that we needed to make this one feature work that we wanted  to pull out of the main app.  We also sent an email saying, hey, we're going to moderate.  We're probably going to break stuff.  We're sorry.  We're going to buy a cake if we break master.  Yeah, it's just a nice thing to do.  So then we said, OK, we got the team.  We got the time.  We have the PMs on our side, and we're allowed to do this.  Let's try it.  And pulling stuff out initially, it  doesn't always go that well.  You move some files, and you're like, oh, look, it's all nice.  And there's no squeak-lea-lands anywhere.  And then you try to compile or run in it.  And yeah, it's a struggle.  So then we tried it again.  Then we tried it again.  And then after a while, it did actually compile.  And we did manage to get that one feature out.  And you have the sort of side goal  that we had on running Unitess in less than five seconds.  That was sort of like, we'll pull it out  and see if it's possible.  And it was.  We didn't have enough to do anything.  So yeah, if anyone were to use that feature  and do more development on it, they could use TDD  and not have to wait for three minutes for Unitess to run.  So through this trial and error in the beginning phase,  we came up with this process.  And I like talking about process through pseudocode  algorithms.  So I'm going to go with that.  So the first thing you want to do  is you want to analyze dependencies of anything  that you want to move.  We have the scale for dependencies,  from frictionless to massive friction.  Where frictionless, that's classes that you can just  move them.  They have no dependencies to anything.  If you move them up, they still work.  And then we have a scale of slight friction to more friction  where you have a class which depends on a class that's  frictionless to move.  Or you depend on a class that depends on another class  and that in turn depends on a class that's frictionless to move.  So there's sort of an infinite scale  of things between more friction and massive friction.  But the massive friction, that's like the main source  of any spaghetti code in the world.  In our case, one of those is main activity, which is,  let's start it out.  I'm sure it was very small at one point.  It's not.  It basically knows about everything  that's going on in the entire app.  There's basically no way in hell that would ever not  live in the main app module.  So yeah, that's the scale.  So when you've analyzed your dependencies and you know,  OK, this is the dependencies, then you  go a few number of ways.  If it has no dependencies, you just move it.  If it asks, yeah, remember tests.  We have a lot of abandoned tests where we've moved out code.  And then we forgot about the test because they never show up  when you analyze the dependencies.  And then they get sad.  I mean, they're still in the app module, so they still run.  But it's nice to have code in the same place.  Yeah, if you have simple dependencies,  you can just break those dependencies.  And simple dependencies here could be something like you're  depending on a class, which is like a utility class.  And I mean, it's nice with don't repeat yourself  and having multiple codes over the same things.  But you could probably inline that.  And it wouldn't be that big of a deal.  Other simple dependencies could be you  might have a dependency on something  because you use it as a parameter.  But maybe you just send that through to somewhere else,  which actually needs the thing.  And then you could just maybe switch that parameter  to something else or just use something else  or do something else.  Maybe the code is old and you can just use a new API  or something.  So then you break the dependency and then  you move on with your life.  If you get into the complex dependencies,  then you need to spend a lot more time.  So then you basically need to spend a lot more time doing  the refactoring.  Or you could try doing dependency inversion.  And I'm going to talk a little bit more about that later.  And if you're so successful with breaking  your complex dependencies, that's great.  But probably you won't be successful.  And one thing we did was we introduced this just very  simple annotation called Moderization To Do,  where we just put that in the code base  where we ran into a wall and we couldn't do anything else.  We added a Moderization To Do and then the reason for it.  And then that makes it just very easy in the code base  to just keep track of where we need to do stuff.  And then revisit that.  Like if you must align, maybe we've already  modernized the bigger dependency that we couldn't  move because of.  But I mean, it's sad.  You spend a lot of time.  So you might stop quite early in the corner  before moving on with your life.  Because the main reason and the main lesson  that we learned really early on in the process  is modernization takes time.  And you need to be patient with modernization.  But like I said, we managed to get that first feature out.  We managed to get more and more features out.  We pulled out more and more code in order  to just make the app as small as possible.  We did have a time in the beginning  where it was just like anything just like,  let's get it out here.  Just standing with shovels.  And yeah, we were just keep doing this and doing this.  And we were keeping metrics of all of this.  And we were just seeing like, we're moving so much code,  but it's not getting smaller.  What's happening?  All right, we're still shipping features.  That's right.  There's still adding stuff.  So it's, you know, it's tough.  So then we're like, hey, can't you just add your code  outside of the main app instead?  Because then the main app doesn't get bigger.  But they couldn't because we didn't have,  we hadn't modernized this one API yet.  Or like this one thing yet, because that's still  in the main app.  So unless you move that, we can't do anything at all.  And that's also started out as thinking,  so that's not really the case.  Maybe you can't move your code immediately or create it  in another place immediately.  But you can write it as if you had it in a module.  Because if you write your code as if it's in a module,  if you then put it in a module, it will be a lot easier.  So this leads to best practices.  And some of the best practices that we developed and talked  about throughout this process, I'm going to share with you now.  So one of them is sort of, don't repeat yourself  from moving dead code and increasing reuse.  And my main point here is this kind of touching on it.  And it's you're probably doing too much in your code.  And you can probably do it in a more organized way  if you think about it as, what things am I actually doing?  Am I doing, I'm probably talking to a backend somewhere  to get some data or doing something like that.  And that's often coded that is repeated.  And you do it in maybe different ways,  in different features, or in different places of your code.  But what you could do instead is just to have one API for this  is how we do HTTP calls.  And then that implementation can be the same and shared.  And that logic doesn't have to live everywhere in the code.  Another one, which I touched on, is avoid parameter dependencies.  So an example for us is we have a feature flagging system  that allows us to do A-B testing and graduate rollouts  and stuff like that.  And what we have to check that is we have this flags object.  And we do a lot of A-B testing, which  means that flags exist basically everywhere.  And existing classes that don't really care about flags  or they're not using them, but it needs to be there  because somewhere someone's using them.  And this is what I was talking about,  that we're using these flag objects to just check  if something should be on and off.  And instead of sending flags all over the place,  we can just send this as a Boolean.  And then we don't have dependencies anymore.  Also makes the code a lot more readable  because rather than seeing flags absolutely everywhere,  you have Booleans going like, should this button be green  or not?  That's just easier to read.  And that ties into dependency injection as well.  And having dependency injection means  that rather than passing around flags  because somewhere needs it, if you're  using dependency injection, you can just  inject flags where you actually need the flag  and where you want to do that check  and not just have it all over the code base.  Structure your code.  And I realized, I rewrote this.  I had this as change the structure of your files  and whatever.  And then I changed it.  And I realized that this is how should you structure your code?  Basically in any way, having structure  is better than not having a structure.  And having a structure that's similar across features is good.  And that doesn't really matter if you're structuring it.  You have UI in one folder and models in one folder  and helpers in one folder.  Doesn't really matter if you have that or something else.  But if it is consistent across features,  it helps out along the way.  It ties into the stuff I was talking about with having  clear APIs for the kind of stuff that you're doing  and knowing what you're doing in your code.  Because if you structure it in UI here and then HTTP here  and then this here, then you might  see that, oh, we're doing a lot of custom HTTP logic where  we could just use this more simpler interface over here  instead and not have to do all of this ourselves.  Avoid using utility classes and static methods.  And this ties in again to the parameter dependencies  because at least at Spotify, we had a lot of utility classes  and a lot of methods.  And what happened with those is because they're static,  obviously, and then you just send in  all of the parameters, just all of them.  Then maybe something changed in the utility.  Maybe someone introduced an A-B test  and something needed to change.  So then all of a sudden, everywhere needs flags all  of a sudden.  And again, just having objects instead,  like helper objects that inject these things that  might not be required for everything everywhere to use.  But this helper class needs to know about flags.  Then you can just create an object,  inject the things you do need, and have the parameters be  just the things that actually is needed from the thing that  is using the helper.  And then solid principles.  And solid principles, I hope most of you know about them.  I'm going to run through them.  This is our TLDR version of them.  Don't do a lot of things in one class.  Design the API so that a class can be used and extended  without being modified.  Don't let implementations of the same interface  behave differently.  Don't design big interfaces.  And depend on abstractions, not on implementations.  And this last one, which is dependency inversion,  this has helped us immensely with modernization.  And yeah, basically what it is is,  instead of having your class directly depend on or use  method of another class somewhere,  you put an interface in between.  And this is very helpful in those examples  where you have massive friction to be able to move  whatever you're using.  So for instance, if you're, in our case,  if we have anything that relies or depends on the main app  or the main activity, instead of trying  to move the main activity, we just make our code dependent  on one interface, which could be like navigate to home.  You often, it can be navigation, but it could be anything.  And then you just have a simple interface, which  has no dependencies whatsoever.  And then you depend on that.  And then if you're using dependency injection,  you can bind the actual implementation of it,  which might be something like main activity  or just a class that uses main activity.  And then you bind that.  So you're injecting the interface.  You have no idea about main activity,  but under the hood, it's still calling main activity.  So what have we actually done up until this point?  So modernization for us started just over a year ago.  And since then, we've moved a lot.  So I don't know if I mentioned metrics,  but we were collecting metrics all throughout our modernization  phase.  And the reason the graph is smooth at one point  is we don't have a lot of data points for that,  because we didn't need to, because we weren't changing  anything.  But I think you can basically see where  we started modernization, which was around,  I don't know if this is visible at all,  but around beginning of 2017.  And this is the period I was talking about where you can see.  And at that point, the app, the main app module  around that stagnant point there,  was around 600,000 lines of code.  And yeah, you can see that stagnating out,  but you can also see the amount of code in the app  growing like crazy.  But the main app is still stagnant.  And that was the bit where we were just shoveling out  water of the sinking ship and then feature developer,  sort of just pouring in more and more and more.  And then something that's interesting with this  is at the end of 2017, you can see the main app module starting  to quite slope and actually decrease properly  for the first times.  And that was basically when we in the modernization team  stopped working on modernization.  So this was when we got to a point  where we've modernized as much things as possible.  It's now on you, the feature developers,  to move out your stuff.  And at that point, we've gotten to a place  where they could do that.  So we got people to do it.  And now it's down to, it's actually  below 300,000 lines of code right now.  It's almost 290,000 lines of code in our monolith module  still, which is basically back to when we were three years  ago in 2015.  So that's the journey, that's the graph to show in our journey.  But it's still very painful.  I'm going to show you one more graph.  And in this graph, the black line at the bottom,  that's the red line here.  So that's the total lines of app in our code base.  Does anyone have a guess of what this green line is?  This is our files.  This green line is the lines of code of our files  that we have.  And we have 25 million lines of our file code.  And this is, yeah, we have around 400 modules at this point,  slightly over 400 and growing.  We have over 14,000 r-files.  And the main reason for this is if you have a module,  you have an r-file.  If you have another module, you have an r-file.  If you depend on that module, you have three r-files.  Because this module will have its r-file,  and that will have its r-file.  But this will have an r-file of that module as well.  And this is something that Greta a lot aware of,  and they're trying to work on and improve for us.  It's not a problem in Basel or Buck.  But we're not using those, so it's painful.  So yeah, let's talk about build times.  And now there was a talk earlier today,  and they went into a lot more numbers about build times  with modernization.  But basically, if you have a smaller module,  it's faster to build and compile incrementally.  But the bottleneck in your build times  will always be the size of your main app module.  And now you have stuff like Basel and Buck,  which are other build systems.  Basically, it's built by Google, Buck is built,  and maintained by Facebook.  And Basel and Buck are a lot better at handling builds  when it comes to a modernized code base,  given that you don't have big modules.  So I had a colleague actually who tried out Buck  and using OK Buck at Spotify, and the build times for the modules  became stupidly slow, like two, three seconds.  But then the main app was like 15 minutes.  So we have a ways to go.  So yeah, I'm going to move on to what's the biggest lessons  and the biggest takeaways that we've learned throughout this  over a year-long process.  And I kind of like this quote.  It's hard to see the forest for all the trees.  It kind of relates to there's this idea called the broken  window theorem.  You may have heard of it, which is if you have an old building  and you have one, if you have no broken windows,  you will not have more broken windows.  But if you have one broken window,  all of a sudden you'll have like 20 broken windows,  because there's already one, and they're not fixing it,  so we can just break everything else.  And why I like this and why I bring it up,  talking about monetization, is that it sort of goes hand  in hand with having a big module and all the code  at one place.  You can't really see all the tech that you have  and all the spaghetti code you have.  And you can't really tell if you're  introducing code that increases the build time by 10 seconds.  If you have a build time that's already minutes long,  maybe you won't even notice that.  But if you're introducing that in a module where the build  time is 10 seconds, and then you're adding 10 seconds,  then all of a sudden it's double, like, that's 200% or 100%.  So if you're in that place, it's easier  to see when problems pop up.  And that's how I see monetization.  I see monetization as a tool to get your code base and your code  to be healthier.  And to make it easier to know what dependencies you have  or what the code quality is.  It's a chance to tackle your tech depth.  Like I was talking about with the best practices,  because everything is small and encapsulated,  you can't really have tech depth in the sense of complex  dependencies and whatever, because you can't have those  because you're in modules.  Or you can, but it's easier not to have them.  And I also see monetization as a chance  to get to know your code base.  And again, this is what I was talking about with getting  to know what kind of stuff you actually do in your code base  and knowing what utilities you use  and knowing what things are in your feature codes  and whether or not that stuff is specific to the feature  itself or the piece of code that is doing stuff  or if it's stuff like technically,  if there was a nice API somewhere, I would use that.  But there's not, so I'm going to build it myself.  And then that happens in 10 different places in the code  base, and then all of a sudden you have a lot more code  than you need to have.  If that is in modules, it's easier to spot those.  And it's easier to think about nicer APIs.  And there's this Yelp article or blog post  that they posted very recently about how they did monetization  at Yelp.  And I thought it was interesting, especially  this first sentence, single modules are easier to deal with.  There are no complex dependency graphs to account for.  And I thought it was interesting,  and I just did revise it, because I think single modules appear  to be easier to deal with.  Because they don't have a complex dependency graph expressed  in build Gradle files, but the dependencies are still there.  The dependencies are still there between the different codes.  But they're hidden because everything  is in this main module.  That ties into this module thing.  If you have things in modules, you  know about the complexity of your code base  in a completely different way.  Let's say some last wrap-up words of wisdom  if you're going to start doing monetization,  or if you're thinking about it, or if you're doing it already.  Investing in some upfront tooling,  like having a script for creating modules,  doing dependency versioning, like having dependency versions  for external third-party libraries in one central place  and not have that copy-pasted in build Gradle files all  over the code base, and then those getting out of sync,  have it in one place.  Also, static analysis for realizing  when you've forgotten resources files, or stuff like that.  Oh, I accidentally left things behind in the main app.  And remember to move your tests.  And don't change anything as you go.  And this is like, you go into your code base,  and you see a file, and you want to move it,  and you look at the code, and it's like, oh,  I can refactor this a little bit.  And it's kind of old code.  Don't do it.  If you want to get monetization done, focus on the task.  Do not clean up as you go.  Don't do anything.  Don't even change the packet structure  if you don't want to.  Just move it out, get it out of there,  focus on the task at hand, add to do's, add it to the backlog  to go back and revisit it.  But do not do it as you go.  You will lose yourself, and you'll never  end up where you want to be.  Inventory your app.  Again, this ties back to just knowing what kind of stuff  should probably be in modules, rather than scattered  around in different feature codes.  Can just help you to reduce the size of the modules  when you already have them.  And then also collect metrics, because graphs are fun.  And then maybe you can go to conferences  and do cool presentations with cool graphs and stuff.  So I'm going to circle back to this initial question  of monetization.  How hard can it be?  Can be very difficult.  And I'm seeing it's a software development.  How hard can that be?  Can also be very difficult.  You can make it easy, or you can make it hard.  If you follow best practices, and you structure your coding  clean ways, and you have nice APIs,  and all the good stuff, it can be easy.  It can be a lovely experience to do software development.  Or you can accidentally turn everything  into a huge ball of spaghetti, and then your life is terrible.  I'm just going to leave you with this.  All I'm satisfying is the channel name of our modernization  Slack channel, where we do modernization support.  And it's basically doing modernization.  It's stressful.  It's tedious at times.  It's very painful.  But in ways, it's oddly satisfying.  You move out stuff, and then you hit a dent end,  and then you cry in a corner for a bit.  But then you revisit it a few weeks later  when someone's moved something else.  And then it's like, oh, look, I can move this now.  And I don't have to worry about it.  After the while, you don't refactor stuff  as you're going along.  But then you do.  You come back to it.  And then you have all this nice, clean code.  And it's just a few classes.  And you can go through them, and everything is nice and readable,  and you can clean it.  And it's just lovely.  Thank you for listening.  [? APPLAUSE?]  And I don't know if we have time for questions.  Well, yeah, we have time for questions.  So please, everyone, use your last chance.  Wow, tons of questions.  Yes, please come here and walk to the mic.  And I'll get the second one for you.  Or you come over there as well.  And everyone else, if you want to leave,  please try to be mindful of the others that still  want to listen.  So be silent.  Thank you.  Hi.  My name is Julio.  I work at SoundCloud.  And first of all, I want to thank you very much for the talk,  which I think was one of the best talk of the conference  up to this point, the last talk also.  So I'm very curious about, after 400 modules you said,  I'm curious about two things.  How is your team organized?  Like, how many developers you have in general?  And how many are working in modularization specifically?  Or if everyone is working in modularization?  And then number two is, what do you think,  or what's your advice for the base module?  What should be there?  OK, well, the first one.  We have over 100 monthly active contributors  to our Android code base.  And we're organized in over 60 teams  in five different cities around the world.  We have New York, Boston, London, Stockholm, and Gothenburg.  So it's a lot of people.  And the modularization team, when we were starting out,  it started with three or four or five.  And then it grew.  So we were eight or nine at the initial points.  But then as we've gone, it's more like teams  are modernizing what they own.  Like, they're modernizing their own code.  As for the other one with the base module, actually,  like I said, I won't be talking about Instant Apps  because we don't have Instant Apps.  We don't have a base module.  So I have no idea.  I would actually like to know myself.  Yeah.  Thank you.  Hi.  Thanks for the awesome talk.  One concern I also have in my project  where I have about 20 Gradle modules is the R.java files  as well.  So one small note, I've watched a talk on Google Iow  this year by the Android Build System team.  And they announced that they are working in it.  So hopefully one day that won't be any such an issue  like now it is.  And I have a question regarding the domain layer  or the model, like the plain model objects.  For instance, you have a song model.  Do you share it among the other modules?  Is it separate?  Is it moved to some kind of a lib model or something  like that module?  Or do you have your own models per every module?  And you have to reassign all the data  when passing this object through other modules.  Yeah.  So yeah, we have actually great contact  with the great old team.  So we actually know about it as well.  And we are hoping that they will do it soon.  As for the other question, I'm so tired already of Gradle.  It was about the modules.  Oh, yeah.  Yeah, models.  Yeah, sorry.  Should go home soon.  Yeah, with the models, it depends.  We have some modules that are definitely  shared between different features  and those we have in some libraries, or in common shared  code.  But there's also cases where a feature actually  has their own models because they're not really  sharing it.  So in that case, it goes into that module.  OK, thanks.  Hello, I'm Mario from N26.  My question was about static analysis.  You mentioned it.  Are you using any libraries out there?  Or what kind of tool are you using for static analysis?  To kind of check that while moderating something,  you just did something, for example.  Yeah, so we're using lint, PMD, find bugs, check style.  Those are the main ones.  We also have one that we've built ourselves,  which is called lift lint, which is lint only on the stuff  that you've changed.  And the main thing there is lint will find all the unused  resources and complain about that.  And it does complain a lot.  And then one thing that I found, which I think it was find bugs,  which so the thing when you moderate stuff,  it can build and run completely fine in some cases.  But then at some point, it just breaks down.  And you have no idea why.  So we have this thing where a lot of just really strange problems  that popped up when we moved files came up through our CI.  And a lot of that was actually caught by find bugs that said  that, oh, something can't compile.  And it made absolutely no sense, because the file it was  complaining about actually existed, but not something  that was dependent on that.  So that helped in knowing that something's not right here.  But it didn't really help with pointing out exactly what that  was.  I think if there was more tooling for figuring these things,  like we created a document with the most common.  If you get this thing, it actually means this.  So we have this translator for all the errors  that we got through built errors.  So tooling could be very helpful if it existed.  And the last question.  Hi.  So yeah, question.  First, in your features model, do you have also  the activity and the Android stuff?  So you manage with the Android to do the dependency  injection in the modules.  So a lot of our features are actually fragment based.  So the fragments we can move out.  But we can have activities outside.  But then with the activity components,  that gets a little bit hairy.  And yeah.  OK.  Because I solved it with Coint.  But Ricardo from SoundCloud told me that the Android  apparently does it.  One more question.  Resources, like the strings, do you  have them in the main module?  Because I try to put the strings in their own module.  But the localization team kind of wanted to kill me.  Because a million different strings file.  Yeah.  Yeah, so we do have one main strings module.  And that has translations.  And we set up our translation scripts region smartling.  And we've set all of that up for that specific module.  But we also have some strings that are actually  in the modules themselves.  If they have strings that are, you know,  they only exist for their own use.  Because are the general strings files,  that's like it's a lot of strings.  That file is very, very long.  If you can avoid being in it, it's nice.  So then people have their own strings  and their own translations within their own modules.  So you wire the translation system to every single module.  Yeah, we have a script that helps out with it.  Perfect.  Thank you.  OK.  So thank you.  I know there are still some questions left.  And I hope, Ellen, you have a couple of minutes  for private questions afterwards.  Yes, I also have stickers.  And stickers.  Wow, OK.  Thank you, everyone, for saying until the end.  Thank you.
_DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_0_587.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_588_2351.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_2352_3095.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_3096_3611.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_3612_5111.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_5112_5411.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_5412_5663.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_5664_6635.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_6636_7919.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_7920_8303.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_8304_9239.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_9240_11447.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_11448_11999.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_12000_12203.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_12204_12539.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_12540_12659.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_12660_13427.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_13428_13871.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_13872_14591.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_14592_15383.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_15384_15635.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_15636_18887.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_18888_19067.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_19068_19847.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_19848_19907.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_19908_19955.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_19956_20015.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_20016_20075.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_20076_20135.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_20136_20207.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_20208_20279.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_20280_20327.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_20328_20387.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_20388_20459.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_20460_20531.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_20532_21635.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_21636_28763.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_28764_31391.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_31392_34895.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_34896_35051.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_35052_35099.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_35100_36863.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_36864_36935.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_36936_37007.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_37008_37067.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_37068_37115.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_37116_37199.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_37200_37247.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_37248_37319.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_37320_37367.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_37368_37427.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_37428_37475.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_37476_44543.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_44544_47327.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_47328_49727.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_49728_62159.png _DCBerlin18_1111_Nilsson_MODULARIZATION_HOW_HARD_C_opsiSaUS2bg_62160_62171.png