Data Visualisation
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Johnston, Neal
GeoBI: A data visualisation platform using D3, leaflet, Geoserver, PostGres and php
 Good day everyone. I'm Neil, Christchurch New Zealand, so another Kiwi right here. Bit of a Kiwi overwrite at the moment, but that's fine. So I'm here to talk to you about one of the projects I've been working on. So I've got a bit of an agenda. I'm going to do a quick introduction to our company because they want me to do that. And the project, the challenge, why I chose open source, the solution and tech stack, and a quick look into the design and then I'm going to try a bit of a demo. So I'll try my luck there. Can people out there back here me all right? Sweet, that's a good start. Just to put it out there, I'm not a big fan of Max, so if this screws up somehow, blame Apple. We'll go from there. Also, the other developer who worked this on me presented this at the Garisal Barmon. He had 25 minutes, I got 15, same content. So I'm just going to speed on through. Okay, all the care, we started about 18 months ago, just being my boss, and we've gone to I think 22 people now. Our purpose is pretty simple. We just try to create and inspire geospatial champions. And that's pretty much the gist of it. Okay, so on with the actual project. This was the aim basically was regional council rates and budget data. I'm trying to show that out to all the ratepayers and provide transparency about what's going on with their rates. In context, I'm from Canterbury, Christchurch in particular. It's one of the largest territories. Contains Christchurch, which is the third largest after Kravya Auckland and Wellington. So still one of the best. The challenge basically, they needed something to be easy to use and interactive and find a nice experience for the users, increase engagement and understanding through their transparency of the data, explain these complex portfolios and programs and again an easy to use way, and be cloud hosted and allow for multiple years. They said multiple apps are sweet, so I went with that. Why open source? Integration is coming from a lot of different departments. Some GIS, some not GIS. We were able to substitute the license cost instead of using the license cost. We used it to play. It gave us freedom, again low implementation costs and a scalable and awesome community. So basically the solution is just a tool that allows the councils to share and update their rates information and it's a nice way to visualize for the residents to see spatial and non-spatial data in the same platform, as well as a narrative that helped them along with the process as well. Technology stack, a whole bunch of stuff. QGIS was made for a couple of them, playing with data and all the others kind of just made their way in there. The solution workflow, that was basically it. So as I said, like the CSVs kind of came from that financial department. GIS gave us shapefiles and the comms and marketing were able to modify the application itself to look and feel the way they wanted it to feel and update with these stories. And all of that pumped through to our PostgreSQL database and out through GOServer into the application. This is just a quick slide on showing that little transformation in the design, I guess. So it's went through quite a few iterations and that was basically it, I think it comes to an end there. All right, the tricky stuff. So I'm going to attempt to try and demo this live. I do understand what that means when it doesn't work. All right, yes. Damn it! What's happening here, man? This is not my thing. I don't know. You all can get it. You turned it off. Yay! So this is the site here. They've got a little narrative down the side. Scrolls don't work in Macs. So they basically can enter in their own stories there. It gives you a little breakdown of the rates information. They've got general targeted grants and other. So this is an overall view. This is all their projects that ECAN is involved in. And down here we have their portfolios. So if we click on one of their portfolios, we can see that it's broken up into programs. So we can come here and investigate each bubble and see what it costs, where all the money is going. And if we click here, it's going to give us a little break down as well as that actual narrative that I was talking about before. And it's cool when you kind of smash through them pretty fast. They smash into each other. It's a bit of fun. Matview. Very simple. This is just a quick way of showing people what's exactly going on. I believe this one's got a nice boundary. A lot of them are general rates so you get just a blank New Zealand thing. So that's not so crazy. Let me find a better one. There we go. So again, this updates our chart over here. It gives us information and we can check out the individual values and where that money is coming from. And they've got a little section in here. They wanted to say, okay, type in my address. That's my house. Please don't knock on my door. And we can go ahead and look at the individual things that rates if I actually own my own house what I would be paying. Unfortunately, I'm still on that track. So we can go here and see exactly where our general rates are going, what percentage each section comes out. And it's broken down into the different portfolios. So that's the application itself. Well, as I said before, they wanted multiple applications. So basically we built CMS behind it so they can come through and create their own app. I'm going to try and do that now without screwing it up. So go ahead, just check make sure that apps not in there. Yes. And we upload a CSV. Oh, no. Desktop. Yes. Good old desktop. CSV. There we go. And we can go ahead and auto populate it from other apps, which I'll do because it's a lot easier. Scrolls the wrong way around. And we can go in there, chuck that in. And all of a sudden it just populates our things. So basically we're mapping our tiers of information to the different sections. And we can go down here when I scroll the opposite way. And it works down this way. So quickly fill these out. Sorry, apologies for me speeding through. I'm trying to put it through as quickly as possible. And yes, that was Pac-Man. Again, we can pop up there. We immediately get a little bit of a visual for what's going on with our information. And I'll go ahead and just quickly pop these in. Step around with that. Make it a little bit easier for myself. It's like I'm just sticking a couple of bits of information here. These are just the generics CSV that I've got. And I'll go to test one. So these are basically just the pop up information for our little pop up over here, which is going to be the hover on our pop up on the bubbles. So we submit that. Yeah, again Pac-Man. Here's a popular guy. Then we start uploading our shape files. So we've got one for more, I guess, specific boundaries and then more your regional boundaries. I like to call this the big aggregator because that's literally what it does. And then we can come through and start selecting our boundaries, design a map. Oh, scroll is awful. Why do people buy these things? So we can set up in a max zoom as well, which limits it on the map as well. So here's basically what we've got. So we can zoom through all that, submit, away it goes. And we do have two different things going on that we've got a nice little summary table that I quite like. And again, it's just another aggregator and another way of displaying a whole bunch more information. So submit that and away it builds our application. Now we're going to delete that because that's a pain in the ass. So away we go. This is essentially what we call our development. So we can come through here and start changing a few things. If I want, I can edit some text, add some story, not a great story writer, so that's all I'm adding. And I can come through, oh man, the scroll is awful. Okay. So I can come through here and decide to change my colors. I can change my chart colors if I want to as well. And there's a whole bunch of icons. I just, sorry, I don't have any pictures. Otherwise I'd upload an icon. And we simply just hit save. And I can push that to our test site, which I have pre-prog, oh great. That's not the next one. No, that one is not. I know why. It's trying to get me to log in again. I'll just log in myself. Apologies. Ah yes, go Google. Okay. Oh no. If I reload this, we should, oh no. Come here. And there's my colors changed. So if I do that again, come through here. I'll do a nicer one. I can change to whatever that color I feel is necessary. That's a beautiful color. So I'm going to save that one and push to my test site. Which I can now just reload. And colors changed. Beautiful. So that's essentially it. It's very straightforward. We try to make it as easy as possible to come through. We can also go ahead and like start moving around titles and things like that. And you know, if you want to change bubbles, kind of, oh man. This mouse is so touchy. All I want to do is go up. Okay. All right. Okay. So we can, you know, slightly move our bubbles or whatever. So if the app looks ugly, it's usually the comms and marketing people's fault. Because we didn't want to do that kind of thing. To be honest, you know, even though we're a consultancy like to make money, I don't want to make money doing this kind of thing. It bores me. So it's much easier just to hand that over to them and let them go crazy and do whatever they want. And they do. So each individual one, they can put stories down to the level of their programs right from, you know, high level story all the way down. So that is basically that lot. If I can go back to this presentation. Yes. Okay. So basically the road ahead was this is kind of, we're looking at developing this into more of a visualisation platform for all the regional councils. So we have a common platform for all of New Zealand. You go to a different region. Same thing. You understand what exactly where all your rates are going. Other regional councils start to get in. All right. So that's great. And we are pushing past just rates because rates get kind of boring after a while and you want to throw something else in there. So we're playing with like throwing our companies data in there and looking at the projects. Boss said don't put that up. So I won't. But basically creating that unified looking field across all the different visualisations. So we've done a bit of work. Currently it's not on mobile, but we are pushing that out. We kind of done a, I guess, a V zero, which is great, which will be pushing out at some stage early next year. So that'll be a bit of fun. That's basically all I got. Is anybody got any questions? Other than the bubbles in the map. Yeah, yeah. Well, that's the plan in the long run. Again, we're being a consultant. So we can bootstrap the thing, get it up there. But we're kind of hoping to start switching it out and making it a bit more open-play for us. So is a lot of you work around this platform here specifically or is this one of the examples of the sort of policy building? Yeah, this is just one of the many things that we end up doing. To be honest, it's the one they kicked off because I guess people like smashing stuff together. But yeah, we do have a range of stuff. We've got a massive range. We just like to play with tools really. How long ago was it idea to launch the customer? And are you in the future looking at how you can take on the comments on community things? Yes. Yes. So basically it was quite a long process. I guess if you're a consultant, you know that generally things don't go 40 hours a week on one thing. So it's extended our company about six months. I'm not sure if I can discuss the exact details about the price part. We certainly got it. It wasn't what you'd expect if we did certainly get quite a reasonable deal out of it. I mean I guess for that, I think like subscription costs? In terms of this, I believe we're looking for future as a subscription model. Just keep that price down low for a lot of the different councils. Especially we've got a lot of smaller councils. They can't really afford that much but we really want to get that out there to them. So we're still in that exploratory stage. But yeah, again we have done a lot of I guess demonstrations with the community and got a lot of feedback from there. But yeah, always let them come home. See you all next time.
 Good day everyone. I'm Neil, Christchurch New Zealand, so another Kiwi right here.  Bit of a Kiwi overwrite at the moment, but that's fine. So I'm here to talk to you about  one of the projects I've been working on. So I've got a bit of an agenda. I'm going  to do a quick introduction to our company because they want me to do that. And the project,  the challenge, why I chose open source, the solution and tech stack, and a quick look  into the design and then I'm going to try a bit of a demo. So I'll try my luck there.  Can people out there back here me all right? Sweet, that's a good start. Just to put it  out there, I'm not a big fan of Max, so if this screws up somehow, blame Apple. We'll  go from there. Also, the other developer who worked this on me presented this at the Garisal  Barmon. He had 25 minutes, I got 15, same content. So I'm just going to speed on through.  Okay, all the care, we started about 18 months ago, just being my boss, and we've gone to  I think 22 people now. Our purpose is pretty simple. We just try to create and inspire geospatial  champions. And that's pretty much the gist of it. Okay, so on with the actual project.  This was the aim basically was regional council rates and budget data. I'm trying to show  that out to all the ratepayers and provide transparency about what's going on with their  rates. In context, I'm from Canterbury, Christchurch in particular. It's one of the largest territories.  Contains Christchurch, which is the third largest after Kravya Auckland and Wellington.  So still one of the best. The challenge basically, they needed something to be easy to use and  interactive and find a nice experience for the users, increase engagement and understanding  through their transparency of the data, explain these complex portfolios and programs and  again an easy to use way, and be cloud hosted and allow for multiple years. They said multiple  apps are sweet, so I went with that. Why open source? Integration is coming from a lot of  different departments. Some GIS, some not GIS. We were able to substitute the license  cost instead of using the license cost. We used it to play. It gave us freedom, again  low implementation costs and a scalable and awesome community. So basically the solution  is just a tool that allows the councils to share and update their rates information and  it's a nice way to visualize for the residents to see spatial and non-spatial data in the  same platform, as well as a narrative that helped them along with the process as well.  Technology stack, a whole bunch of stuff. QGIS was made for a couple of them, playing  with data and all the others kind of just made their way in there. The solution workflow,  that was basically it. So as I said, like the CSVs kind of came from that financial department.  GIS gave us shapefiles and the comms and marketing were able to modify the application itself  to look and feel the way they wanted it to feel and update with these stories. And all  of that pumped through to our PostgreSQL database and out through GOServer into the application.  This is just a quick slide on showing that little transformation in the design, I guess.  So it's went through quite a few iterations and that was basically it, I think it comes  to an end there. All right, the tricky stuff. So I'm going to attempt to try and demo this  live. I do understand what that means when it doesn't work. All right, yes. Damn it!  What's happening here, man? This is not my thing. I don't know. You all can get it.  You turned it off. Yay! So this is the site here. They've got a little narrative down  the side. Scrolls don't work in Macs. So they basically can enter in their own stories there.  It gives you a little breakdown of the rates information. They've got general targeted  grants and other. So this is an overall view. This is all their projects that ECAN is involved  in. And down here we have their portfolios. So if we click on one of their portfolios,  we can see that it's broken up into programs. So we can come here and investigate each bubble  and see what it costs, where all the money is going. And if we click here, it's going  to give us a little break down as well as that actual narrative that I was talking about  before. And it's cool when you kind of smash through them pretty fast. They smash into each  other. It's a bit of fun. Matview. Very simple. This is just a quick way of showing people  what's exactly going on. I believe this one's got a nice boundary. A lot of them are general  rates so you get just a blank New Zealand thing. So that's not so crazy. Let me find  a better one. There we go. So again, this updates our chart over here. It gives us information  and we can check out the individual values and where that money is coming from. And they've  got a little section in here. They wanted to say, okay, type in my address. That's my  house. Please don't knock on my door. And we can go ahead and look at the individual  things that rates if I actually own my own house what I would be paying. Unfortunately,  I'm still on that track. So we can go here and see exactly where our general rates are  going, what percentage each section comes out. And it's broken down into the different  portfolios. So that's the application itself. Well, as I said before, they wanted multiple  applications. So basically we built CMS behind it so they can come through and create their  own app. I'm going to try and do that now without screwing it up. So go ahead, just  check make sure that apps not in there. Yes. And we upload a CSV. Oh, no. Desktop. Yes.  Good old desktop. CSV. There we go. And we can go ahead and auto populate it from other  apps, which I'll do because it's a lot easier. Scrolls the wrong way around. And we can go  in there, chuck that in. And all of a sudden it just populates our things. So basically  we're mapping our tiers of information to the different sections. And we can go down  here when I scroll the opposite way. And it works down this way. So quickly fill these  out. Sorry, apologies for me speeding through. I'm trying to put it through as quickly as  possible. And yes, that was Pac-Man. Again, we can pop up there. We immediately get a  little bit of a visual for what's going on with our information. And I'll go ahead and  just quickly pop these in. Step around with that. Make it a little bit easier for myself.  It's like I'm just sticking a couple of bits of information here. These are just the generics  CSV that I've got. And I'll go to test one. So these are basically just the pop up information  for our little pop up over here, which is going to be the hover on our pop up on the  bubbles. So we submit that. Yeah, again Pac-Man. Here's a popular guy. Then we start uploading  our shape files. So we've got one for more, I guess, specific boundaries and then more  your regional boundaries. I like to call this the big aggregator because that's literally  what it does. And then we can come through and start selecting our boundaries, design  a map. Oh, scroll is awful. Why do people buy these things? So we can set up in a max  zoom as well, which limits it on the map as well. So here's basically what we've got.  So we can zoom through all that, submit, away it goes. And we do have two different things  going on that we've got a nice little summary table that I quite like. And again, it's just  another aggregator and another way of displaying a whole bunch more information. So submit that  and away it builds our application. Now we're going to delete that because that's a pain  in the ass. So away we go. This is essentially what we call our development. So we can come  through here and start changing a few things. If I want, I can edit some text, add some  story, not a great story writer, so that's all I'm adding. And I can come through, oh  man, the scroll is awful. Okay. So I can come through here and decide to change my colors.  I can change my chart colors if I want to as well. And there's a whole bunch of icons.  I just, sorry, I don't have any pictures. Otherwise I'd upload an icon. And we simply  just hit save. And I can push that to our test site, which I have pre-prog, oh great.  That's not the next one. No, that one is not. I know why. It's trying to get me to log in  again. I'll just log in myself. Apologies. Ah yes, go Google. Okay. Oh no. If I reload  this, we should, oh no. Come here. And there's my colors changed. So if I do that again,  come through here. I'll do a nicer one. I can change to whatever that color I feel is  necessary. That's a beautiful color. So I'm going to save that one and push to my test  site. Which I can now just reload. And colors changed. Beautiful. So that's essentially  it. It's very straightforward. We try to make it as easy as possible to come through. We  can also go ahead and like start moving around titles and things like that. And you know,  if you want to change bubbles, kind of, oh man. This mouse is so touchy. All I want to  do is go up. Okay. All right. Okay. So we can, you know, slightly move our bubbles or  whatever. So if the app looks ugly, it's usually the comms and marketing people's fault. Because  we didn't want to do that kind of thing. To be honest, you know, even though we're a  consultancy like to make money, I don't want to make money doing this kind of thing. It  bores me. So it's much easier just to hand that over to them and let them go crazy and  do whatever they want. And they do. So each individual one, they can put stories down  to the level of their programs right from, you know, high level story all the way down.  So that is basically that lot. If I can go back to this presentation. Yes. Okay. So basically  the road ahead was this is kind of, we're looking at developing this into more of a  visualisation platform for all the regional councils. So we have a common platform for  all of New Zealand. You go to a different region. Same thing. You understand what exactly  where all your rates are going. Other regional councils start to get in. All right. So that's  great. And we are pushing past just rates because rates get kind of boring after a while  and you want to throw something else in there. So we're playing with like throwing our companies  data in there and looking at the projects. Boss said don't put that up. So I won't. But  basically creating that unified looking field across all the different visualisations. So  we've done a bit of work. Currently it's not on mobile, but we are pushing that out. We  kind of done a, I guess, a V zero, which is great, which will be pushing out at some stage  early next year. So that'll be a bit of fun. That's basically all I got. Is anybody got  any questions?  Other than the bubbles in the map. Yeah, yeah. Well, that's the plan in the long run. Again,  we're being a consultant. So we can bootstrap the thing, get it up there. But we're kind  of hoping to start switching it out and making it a bit more open-play for us.  So is a lot of you work around this platform here specifically or is this one of the examples  of the sort of policy building? Yeah, this is just one of the many things that we end  up doing. To be honest, it's the one they kicked off because I guess people like smashing  stuff together. But yeah, we do have a range of stuff. We've got a massive range. We just  like to play with tools really.  How long ago was it idea to launch the customer? And are you in the future looking at how  you can take on the comments on community things?  Yes. Yes. So basically it was quite a long process. I guess if you're a consultant, you  know that generally things don't go 40 hours a week on one thing. So it's extended our  company about six months. I'm not sure if I can discuss the exact details about the  price part. We certainly got it. It wasn't what you'd expect if we did certainly get  quite a reasonable deal out of it. I mean I guess for that, I think like subscription  costs? In terms of this, I believe we're looking for future as a subscription model. Just keep  that price down low for a lot of the different councils. Especially we've got a lot of smaller  councils. They can't really afford that much but we really want to get that out there to  them. So we're still in that exploratory stage. But yeah, again we have done a lot of I guess  demonstrations with the community and got a lot of feedback from there. But yeah, always  let them come home. See you all next time.
Neal_Johnston__GeoBI__A_data_visualisation_platform_using_D3__leaflet__Geoserver__PostGres_and_php_0_431.png Neal_Johnston__GeoBI__A_data_visualisation_platform_using_D3__leaflet__Geoserver__PostGres_and_php_432_659.png Neal_Johnston__GeoBI__A_data_visualisation_platform_using_D3__leaflet__Geoserver__PostGres_and_php_660_1859.png Neal_Johnston__GeoBI__A_data_visualisation_platform_using_D3__leaflet__Geoserver__PostGres_and_php_1860_2855.png Neal_Johnston__GeoBI__A_data_visualisation_platform_using_D3__leaflet__Geoserver__PostGres_and_php_2856_4295.png Neal_Johnston__GeoBI__A_data_visualisation_platform_using_D3__leaflet__Geoserver__PostGres_and_php_4296_4559.png Neal_Johnston__GeoBI__A_data_visualisation_platform_using_D3__leaflet__Geoserver__PostGres_and_php_4560_5303.png Neal_Johnston__GeoBI__A_data_visualisation_platform_using_D3__leaflet__Geoserver__PostGres_and_php_5304_5351.png Neal_Johnston__GeoBI__A_data_visualisation_platform_using_D3__leaflet__Geoserver__PostGres_and_php_5352_5615.png Neal_Johnston__GeoBI__A_data_visualisation_platform_using_D3__leaflet__Geoserver__PostGres_and_php_5616_6107.png Neal_Johnston__GeoBI__A_data_visualisation_platform_using_D3__leaflet__Geoserver__PostGres_and_php_6108_10163.png Neal_Johnston__GeoBI__A_data_visualisation_platform_using_D3__leaflet__Geoserver__PostGres_and_php_10164_10247.png Neal_Johnston__GeoBI__A_data_visualisation_platform_using_D3__leaflet__Geoserver__PostGres_and_php_10248_15119.png Neal_Johnston__GeoBI__A_data_visualisation_platform_using_D3__leaflet__Geoserver__PostGres_and_php_15120_15359.png Neal_Johnston__GeoBI__A_data_visualisation_platform_using_D3__leaflet__Geoserver__PostGres_and_php_15360_16727.png Neal_Johnston__GeoBI__A_data_visualisation_platform_using_D3__leaflet__Geoserver__PostGres_and_php_16728_19571.png Neal_Johnston__GeoBI__A_data_visualisation_platform_using_D3__leaflet__Geoserver__PostGres_and_php_19572_20807.png Neal_Johnston__GeoBI__A_data_visualisation_platform_using_D3__leaflet__Geoserver__PostGres_and_php_20808_20903.png Neal_Johnston__GeoBI__A_data_visualisation_platform_using_D3__leaflet__Geoserver__PostGres_and_php_20904_21827.png Neal_Johnston__GeoBI__A_data_visualisation_platform_using_D3__leaflet__Geoserver__PostGres_and_php_21828_21875.png Neal_Johnston__GeoBI__A_data_visualisation_platform_using_D3__leaflet__Geoserver__PostGres_and_php_21876_21935.png Neal_Johnston__GeoBI__A_data_visualisation_platform_using_D3__leaflet__Geoserver__PostGres_and_php_21936_22967.png