Data Classes and Parsing Json - A Story About Converting Models to Kotlin
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Medack, Stefan
The announcement of Kotlin's first class support at Google I/O 2017 is only one year old and its popularity is growing rapidly ever since. At Kitchen Stories we fell in love with Kotlin and we are migrating our code base gradually to leverage the features Kotlin provides us. One of our first migrations was changing all Models assembling the JSON structure from our API. In this session I want to take you along the journey on how and why we migrated all our ""POJOs"" to ""POKOs"". You will learn how you can write better models using Kotlin's Data Classes and how you can leverage features like constructor default values and non-nullable fields for more robust JSON parsing.
 So hi everybody, I'm Stefan and today I want to talk to you about how we used Kotlin at our company at leverage features while we're passing Jason. So I work at Kitchen Stories which is a Berlin based startup and just to summarize under the slogan everyone can cook we will try to bring for free a multimedia cooking experience to everybody at home and that's where we work right now and I guess everybody has seen last year's I.O. and the biggest announcement of last year's I.O. after that I tried to convince our design team to change the logo to this. They're not quite yet on board but maybe I will manage after we migrated everything to Kotlin because right now we already had 75% Kotlin adoption within one year I guess that's quite good and this talk is going to be about one of our first one of our first migrations. So it starts like every good story starts how can you make your product owner happy. What we have right now and I would just want to summarize it really quick is that we have an API that cannot be so easily changed because our teams quite slim and our back-end guys are really busy but we have a quite complicated API because our content is delivered into 12 languages and then we have a content team translating recipes and articles in-house and sometimes they tend to forget to enter something in the field and what we had because of that there was what we can consider broken content delivered via the API to the app and our app crashed just no point exception because title was null for some reason for an article or a recipe and what we wanted to have is less of those content related questions and at the same time we were trying to get rid of all the tedious null checks like is the recipe null is the title of the recipe null or as the author of the recipe panel and then is the name of the author of the recipe null and you get the just it's it can be like three lines of if statements just to check if you can read a field without having the app crash and we had to write a lot of those code and we still had crashes and every time there was like a big crash we fixed of course the content related bug but they were just a lot so we were trying to solve this problem at an earlier stage and just filled as a spoken content instead of using it within the app so there was our goal and just to wake you guys all up a little it's a little quiz time how many different ways do you think are there to define a string in a JSON and I know the question is quite difficult to grasp so I will just help you out of course you can have a field and it has some text that's just a valid definition that's the first option there's second option that the field is just no of course there's always that as API can say there's no field in this case but there's also a third option and maybe some people tend to forget that that's a few is just not sent and keep that in mind because it's going to be important for the rest of the talk so I'll just dive right into an example a pretty easy example I guess everybody can grasp which is actual an actual example of our API and our content so we have recipes and also articles and articles are fairly simple they have a title they have a subtitle they can have a banner image which we call cell image and then they have an author and then content is a list of items comments is a list of items I abbreviated a little bit too for you to have a better overview and what we had before was this ugly Java class I would say because I didn't put everything on here we still had to define get this and set as for us a field we had to find the constructors and we had to define if we want to compare articles to each other and equals mesh and so on and we simply just converted it to Kotlin and after that it looks like this I can just show you once again because I like the animation but you see every field stays at the same spot and it looks like not a lot has changed but I want to dive into it because I'm not sure if everybody is familiar with the concepts of Kotlin and what we're actually using here so to be on the same page first of all I clean up clean it up a little bit because I don't quite like the snake case so you can easily just rename the field to something else and of course you still have to tell it what the actual field in the JSON later is so that it knows keep that in mind if you do this but for a better overview I will leave it out now and now let's dive into the Kotlin class because it's quite interesting first of all I just told you in Java we had to write all the getters, setters, constructors everything in Kotlin this are four letters data you just write a data class and then it generates for all fields getters and setters and equals method and hash function and also a nice function which is called copy where you can just copy instances and just overwrite values you can say copy with a new title and then you have a new article with a different title but everything else stays the same since we are loading the data from the back end there's no reason for us to change it so we made the fields immutable you could do this in Java too but we just decided to do this change here so this vowel you can easily say this field cannot change and if you want to have changing field it's VAR in Kotlin so one of the really nice features of Kotlin is in a data class with constructor like this you can provide default parameters so when you remember back the quiz options three if our API doesn't send for example a subtitle or no content because there is none then we could reason about it already at the passing it's a passing step and initialize fields with other values if they are missing so if there's no subtitle in Java it passed that as not but there's no real meaning behind having a null subtitle so for us it's just an empty string in this case and it is even better with lists like you can see with content in comment because we don't have nullable lists anymore we just have empty lists so we don't have to check for now before we access the list it's just empty if our API sends no content or no comments and like count is zero yeah with with all those default parameters those fields become more less optional fields our JSON API doesn't have to send those fields we can reason about how to display the content if they are not available those four fields become on the other side mandatory fields because we don't provide default values there needs to be data sent via the API that fill those fields otherwise the whole passing process would crash so we cannot use an article without a title in the app anywhere that's why already at the design at the design phase we decided to have title as a mandatory field and the same goes for image and author and there's one special field which is this one every field is not nullable by default except in Kotlin except you put a question mark behind it so the only nullable field in our model is video so I guess you're all familiar with our models now in part two of the talk I will just show you how you can pass data actually an Android and of course you need to find you could write your own part or you find a passing libraries that works really well with Kotlin doing my research I found three first is Kotlin which has a weird name if you speak German as you just realized Jackson is a quite popular one for backend guys but it's a huge library that's why it's not really recommended for mobile use and then there's Moshe developed by Square and of course at this point I could compare all of them to benchmarks and everything but I will skip this part and just tell you how to set up Moshe two easy steps first you edit to your build dot gradle so the latest version of Moshe is 1.6 when I first built the slides it was 1.5 but I managed to update the slides you just implement Moshe and then you can right away pass Java that's everything you need if you want to leverage all the Kotlin features I just showed you you have two options version a is you add another adapter which is bundled in Moshe Kotlin plugin and in version 1.0 and this adapter adds the Kotlin reflect lobby so again really big library in your small APK or the new version coming with Moshe 1.6 is adding Kotlin code generation and in this case it generates for every class you want to pass an adapter at build time so I would say version B, B stands for better in this case because you don't have to ship Kotlin reflect anymore and A you can count as another deprecated version maybe so let's just get version A out of the equation and work with version B then there's step two set up a Moshe adapter which you use for parsing and okay there's still version A here I just want to show for complete list how you would use it so there's a magic part where can be dragons and rainbows and adapters for your parsing and you just add one adapter which is the Kotlin JSON adapter factory and then it can do all the stuff I just showed you with constructor default values and everything when you pass JSON into Kotlin modules before that without this adapter couldn't do this but like I said it says version B B stands for better you can you don't need to add any adapter if you will use version B you just go back to your model and you say generate an adapter for this that's it and now it generates at build time all the adapters it needs and works quite well so version B doesn't work for you can always go back to version A but if it works prefer version B yeah in steps three I will leave out but it's pretty easy to set up Moshe to use to work with way to a fit or whatever layer for loading the data actually is and we thought we managed to do everything now and that's the end but is it really the end of course not that would be quite short presentation what do you do if you want to verify what I just told you pretty easy question of course you write tests and there's a repo I built where I put all the setup and everything all the models I showed you and even more where you can check out what I described and they are really nice tests you can see on how everything works and while writing those tests we figured out three problems we have to solve one with fairly easy Moshe doesn't know how to translate the date and two more difficult ones I will get to that once a minute later how can we tell Moshe that should pass it how it should pass a date that's pretty easy there's also a plug-in Moshe adapters where there's a if I see and so on Jason adapter and you just add that one to your Moshe adapter pipeline and it knows how to pass a date fairly simple now for that example to understand better I will go back to the article like I told you we have mandatory fields in our article and we don't want to work with an article that doesn't have that so but what happens if one of those fields is missing in the Jason when he's passing simple it's a crash and we have to find a solution how to remove those crashes because we had those crashes before and we didn't want to have them anymore so luckily in in the Moshe examples there's was this adapter I converted to Kotlin there's not so much you need to understand about how it works at the longer presentation I talked about how it works but what it does is fairly simple you add it to your build pipeline again you define the mismatch adapter and say for articles when the passing process crashes don't give me don't crash instead return and now you can but this adapter is pretty pretty flexible you can also use it for example to pass an enum and if it doesn't match anything in your enum you could just say well and instead of crashing just assume aspect ratio is unknown or you can even return like for an article page in our example article page is just a list of articles in an object you can just return an empty article page and this way we managed to get this crash out of the way but then we found another crash which okay which is not really unique to our solution what we are doing but a problem how Moshe in general serializes JSON into models and they are lists that can have that are declared in Kotlin as non-nullable and they can contain nullable stuff so I can easiest describe it with content in comments as you can see content in comment doesn't have this question mark so content and comment are supposed to be not null in the list but somehow Moshe manages to serialize nulls into this list and then later on it crashes when you use the list another fancy adapter it's in the way for you can check it out but what it does is more less the same I added to the pipeline and I say for content just filter out now values out of the list okay and I can do the same for comment and that's it all tests are passing success so a little summary for everybody fell asleep too long fell asleep I will just summarize everything I just said because yeah first of all Kotlin is fun it's awesome it's really really fun and if you haven't used it yet I hope I convinced you to leverage some of the features put Kotlin provides because it is really fun to work with it puts fun back into programming because methods are called fun but also because it is fun but it's a tool so use it as a tool don't stick to Java because you're used to Java use Kotlin as a tool as you would with Java I showed you all the nice features like immutability non-nullable fields and construct the default values and data classes you can all check them out on the Kotlin dogo's fairly simple to understand then I of course I talked about how we pass JSON into one of those models and our key challenge how we filter broken content from what we pass and we found a workaround for the list bug which is actually a real bug in mushy and they don't know how to fix it themselves but with this workaround you can at least work around this and just to keep it complete maybe this version is a solution with the adapters I showed you it doesn't scale really well for thousands of models in this case would be good to also do some auto generation or anything or something like that but I have managed to do that yet and probably won't so you all the links I'm not quite sure how this slide got in there but it's probably important but I'm really thankful for listening I'm looking forward to feedback the only reason I'm sharing this is because we want to have feedback and discussion on how you would do it differently so if you have any questions and understanding or recommendations or whatever you can call you can write me on Twitter you can just come back to the stage I'm not sure if there are time for real questions via Mike or you can submit pull requests to the way poll I would be glad about that as well thank you
 So hi everybody, I'm Stefan and today I want to talk to you about how we used  Kotlin at our company at leverage features while we're passing Jason. So I  work at Kitchen Stories which is a Berlin based startup and just to  summarize under the slogan everyone can cook we will try to bring for free a  multimedia cooking experience to everybody at home and that's where we  work right now and I guess everybody has seen last year's I.O. and the biggest  announcement of last year's I.O. after that I tried to convince our design team  to change the logo to this. They're not quite yet on board but maybe I will  manage after we migrated everything to Kotlin because right now we already had  75% Kotlin adoption within one year I guess that's quite good and this talk  is going to be about one of our first one of our first migrations. So it starts  like every good story starts how can you make your product owner happy. What we  have right now and I would just want to summarize it really quick is that we have  an API that cannot be so easily changed because our teams quite slim and our  back-end guys are really busy but we have a quite complicated API because our  content is delivered into 12 languages and then we have a content team  translating recipes and articles in-house and sometimes they tend to  forget to enter something in the field and what we had because of that there  was what we can consider broken content delivered via the API to the app and our  app crashed just no point exception because title was null for some reason  for an article or a recipe and what we wanted to have is less of those content  related questions and at the same time we were trying to get rid of all the  tedious null checks like is the recipe null is the title of the recipe null or  as the author of the recipe panel and then is the name of the author of the  recipe null and you get the just it's it can be like three lines of if statements  just to check if you can read a field without having the app crash and we had  to write a lot of those code and we still had crashes and every time there was  like a big crash we fixed of course the content related bug but they were just a  lot so we were trying to solve this problem at an earlier stage and just  filled as a spoken content instead of using it within the app so there was our  goal and just to wake you guys all up a little it's a little quiz time how many  different ways do you think are there to define a string in a JSON and I know  the question is quite difficult to grasp so I will just help you out of course you  can have a field and it has some text that's just a valid definition that's the  first option there's second option that the field is just no of course there's  always that as API can say there's no field in this case but there's also a  third option and maybe some people tend to forget that that's a few is just not  sent and keep that in mind because it's going to be important for the rest of  the talk so I'll just dive right into an example a pretty easy example I guess  everybody can grasp which is actual an actual example of our API and our  content so we have recipes and also articles and articles are fairly simple  they have a title they have a subtitle they can have a banner image which we  call cell image and then they have an author and then content is a list of  items comments is a list of items I abbreviated a little bit too for you to  have a better overview and what we had before was this ugly Java class I would  say because I didn't put everything on here we still had to define get this and  set as for us a field we had to find the constructors and we had to define if we  want to compare articles to each other and equals mesh and so on and we  simply just converted it to Kotlin and after that it looks like this I can just  show you once again because I like the animation but you see every field stays  at the same spot and it looks like not a lot has changed but I want to dive into  it because I'm not sure if everybody is familiar with the concepts of Kotlin and  what we're actually using here so to be on the same page first of all I clean up  clean it up a little bit because I don't quite like the snake case so you can  easily just rename the field to something else and of course you still  have to tell it what the actual field in the JSON later is so that it knows keep  that in mind if you do this but for a better overview I will leave it out now  and now let's dive into the Kotlin class because it's quite interesting first  of all I just told you in Java we had to write all the getters, setters, constructors  everything in Kotlin this are four letters data you just write a data  class and then it generates for all fields getters and setters and equals method  and hash function and also a nice function which is called copy where you  can just copy instances and just overwrite values you can say copy with a  new title and then you have a new article with a different title but  everything else stays the same since we are loading the data from the back end  there's no reason for us to change it so we made the fields immutable you could  do this in Java too but we just decided to do this change here so this vowel you  can easily say this field cannot change and if you want to have changing field  it's VAR in Kotlin so one of the really nice features of Kotlin is in a data  class with constructor like this you can provide default parameters so when you  remember back the quiz options three if our API doesn't send for example a subtitle  or no content because there is none then we could reason about it already at the  passing it's a passing step and initialize fields with other values if  they are missing so if there's no subtitle in Java it passed that as not  but there's no real meaning behind having a null subtitle so for us it's just an  empty string in this case and it is even better with lists like you can see  with content in comment because we don't have nullable lists anymore we  just have empty lists so we don't have to check for now before we access the list  it's just empty if our API sends no content or no comments and like count is  zero  yeah with with all those default parameters those fields become more  less optional fields our JSON API doesn't have to send those fields we can  reason about how to display the content if they are not available those four  fields become on the other side mandatory fields because we don't provide  default values there needs to be data sent via the API that fill those fields  otherwise the whole passing process would crash so we cannot use an article  without a title in the app anywhere that's why already at the design at the  design phase we decided to have title as a mandatory field and the same goes for  image and author and there's one special field which is this one every field is  not nullable by default except in Kotlin except you put a question mark behind  it so the only nullable field in our model is video so I guess you're all  familiar with our models now in part two of the talk I will just show you how  you can pass data actually an Android and of course you need to find you could  write your own part or you find a passing libraries that works really well  with Kotlin doing my research I found three first is Kotlin which has a weird  name if you speak German as you just realized Jackson is a quite popular  one for backend guys but it's a huge library that's why it's not really  recommended for mobile use and then there's Moshe developed by Square and of  course at this point I could compare all of them to benchmarks and everything  but I will skip this part and just tell you how to set up Moshe two easy steps  first you edit to your build dot gradle so the latest version of Moshe is 1.6  when I first built the slides it was 1.5 but I managed to update the slides you  just implement Moshe and then you can right away pass Java that's everything  you need if you want to leverage all the Kotlin features I just showed you you  have two options version a is you add another adapter which is bundled in  Moshe Kotlin plugin and in version 1.0 and this adapter adds the Kotlin  reflect lobby so again really big library in your small APK or the new  version coming with Moshe 1.6 is adding Kotlin code generation and in this case  it generates for every class you want to pass an adapter at build time so I would  say version B, B stands for better in this case because you don't have to ship  Kotlin reflect anymore and A you can count as another deprecated version maybe  so let's just get version A out of the equation and work with version B then  there's step two set up a Moshe adapter which you use for parsing and okay  there's still version A here I just want to show for complete list how you would  use it so there's a magic part where can be dragons and rainbows and adapters for  your parsing and you just add one adapter which is the Kotlin JSON adapter  factory and then it can do all the stuff I just showed you with constructor  default values and everything when you pass JSON into Kotlin modules before  that without this adapter couldn't do this but like I said it says version B  B stands for better you can you don't need to add any adapter if you will use  version B you just go back to your model and you say generate an adapter for  this that's it and now it generates at build time all the adapters it needs and  works quite well so version B doesn't work for you can always go back to  version A but if it works prefer version B yeah in steps three I will leave out  but it's pretty easy to set up Moshe to use to work with way to a fit or whatever  layer for loading the data actually is and we thought we managed to do  everything now and that's the end but is it really the end of course not that  would be quite short presentation what do you do if you want to verify what I  just told you pretty easy question of course you write tests and there's a  repo I built where I put all the setup and everything all the models I showed  you and even more where you can check out what I described and they are really  nice tests you can see on how everything works and while writing those tests we  figured out three problems we have to solve one with fairly easy Moshe doesn't  know how to translate the date and two more difficult ones I will get to that  once a minute later how can we tell Moshe that should pass it how it should  pass a date that's pretty easy there's also a plug-in Moshe adapters where  there's a if I see and so on Jason adapter and you just add that one to your  Moshe adapter pipeline and it knows how to pass a date fairly simple now for that  example to understand better I will go back to the article like I told you we  have mandatory fields in our article and we don't want to work with an article  that doesn't have that so but what happens if one of those fields is missing  in the Jason when he's passing simple it's a crash and we have to find a  solution how to remove those crashes because we had those crashes before and  we didn't want to have them anymore so luckily in in the Moshe examples there's  was this adapter I converted to Kotlin there's not so much you need to  understand about how it works at the longer presentation I talked about how  it works but what it does is fairly simple you add it to your build pipeline  again you define the mismatch adapter and say for articles when the passing  process crashes don't give me don't crash instead return and now you can but  this adapter is pretty pretty flexible you can also use it for example to pass  an enum and if it doesn't match anything in your enum you could just say well and  instead of crashing just assume aspect ratio is unknown or you can even return  like for an article page in our example article page is just a list of articles  in an object you can just return an empty article page and this way we managed  to get this crash out of the way but then we found another crash which okay  which is not really unique to our solution what we are doing but a problem  how Moshe in general serializes JSON into models and they are lists that can  have that are declared in Kotlin as non-nullable and they can contain  nullable stuff so I can easiest describe it with content in comments as you can  see content in comment doesn't have this question mark so content and comment are  supposed to be not null in the list but somehow Moshe manages to serialize  nulls into this list and then later on it crashes when you use the list another  fancy adapter it's in the way for you can check it out but what it does is more  less the same I added to the pipeline and I say for content just filter out now  values out of the list okay and I can do the same for comment and that's it all  tests are passing success so a little summary for everybody fell asleep too  long fell asleep I will just summarize everything I just said because yeah first  of all Kotlin is fun it's awesome it's really really fun and if you haven't  used it yet I hope I convinced you to leverage some of the features put Kotlin  provides because it is really fun to work with it puts fun back into  programming because methods are called fun but also because it is fun but it's  a tool so use it as a tool don't stick to Java because you're used to Java use  Kotlin as a tool as you would with Java I showed you all the nice features like  immutability non-nullable fields and construct the default values and data  classes you can all check them out on the Kotlin dogo's fairly simple to  understand then I of course I talked about how we pass JSON into one of  those models and our key challenge how we filter broken content from what we  pass and we found a workaround for the list bug which is actually a real bug  in mushy and they don't know how to fix it themselves but with this workaround  you can at least work around this and just to keep it complete maybe this  version is a solution with the adapters I showed you it doesn't scale really well  for thousands of models in this case would be good to also do some auto  generation or anything or something like that but I have managed to do that yet  and probably won't so you all the links I'm not quite sure how this slide got in  there but it's probably important but I'm really thankful for listening I'm  looking forward to feedback the only reason I'm sharing this is because we  want to have feedback and discussion on how you would do it differently so if  you have any questions and understanding or recommendations or whatever you can  call you can write me on Twitter you can just come back to the stage I'm not  sure if there are time for real questions via Mike or you can submit pull  requests to the way poll I would be glad about that as well thank you
_DCBerlin18_204_Medack_Data_Classes_and_Parsing_Js_0FwBkjvhpoU_0_371.png _DCBerlin18_204_Medack_Data_Classes_and_Parsing_Js_0FwBkjvhpoU_372_1679.png _DCBerlin18_204_Medack_Data_Classes_and_Parsing_Js_0FwBkjvhpoU_1680_4307.png _DCBerlin18_204_Medack_Data_Classes_and_Parsing_Js_0FwBkjvhpoU_4308_4787.png _DCBerlin18_204_Medack_Data_Classes_and_Parsing_Js_0FwBkjvhpoU_4788_15695.png _DCBerlin18_204_Medack_Data_Classes_and_Parsing_Js_0FwBkjvhpoU_15696_16427.png _DCBerlin18_204_Medack_Data_Classes_and_Parsing_Js_0FwBkjvhpoU_16428_17363.png _DCBerlin18_204_Medack_Data_Classes_and_Parsing_Js_0FwBkjvhpoU_17364_17591.png _DCBerlin18_204_Medack_Data_Classes_and_Parsing_Js_0FwBkjvhpoU_17592_20051.png _DCBerlin18_204_Medack_Data_Classes_and_Parsing_Js_0FwBkjvhpoU_20052_22619.png _DCBerlin18_204_Medack_Data_Classes_and_Parsing_Js_0FwBkjvhpoU_22620_23063.png _DCBerlin18_204_Medack_Data_Classes_and_Parsing_Js_0FwBkjvhpoU_23064_23147.png _DCBerlin18_204_Medack_Data_Classes_and_Parsing_Js_0FwBkjvhpoU_23148_25679.png _DCBerlin18_204_Medack_Data_Classes_and_Parsing_Js_0FwBkjvhpoU_25680_25835.png _DCBerlin18_204_Medack_Data_Classes_and_Parsing_Js_0FwBkjvhpoU_25836_26615.png _DCBerlin18_204_Medack_Data_Classes_and_Parsing_Js_0FwBkjvhpoU_26616_26622.png