GeoTools, GeoServer, GeoGit: A Case Study of Use in Utility Field Work
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Hansen, Peter
After creating a custom application featuring ArcSDE and the ESRI Mobile SDK for use by the field crew from a local Mosquito and Vector Abatement District, we sought an alternative to the high overhead from the proprietary software. By utilizing GeoTools, GeoServer, and GeoGit, we were able to develop a full-fledged application maintaining the same functionality and usability of the original application, without the high cost of entry.The GeoTools application, "Mosquito," and GeoServer, were placed on each of the field laptops of the twenty-member crew, serving both the application and cached base layers to allow for offline data connection. A USB Bluetooth GPS dongle was used to allow workers to locate themselves within the application. GeoGit was utilized to allow the disparate field workers to merge and synchronize data to the master database at the end of their shift.
 All right, I think it's about 10 o'clock, so let's get the show on the road. My name is Peter Hansen. I work for the Geographical Information Center at California State University, Chico. I have a presentation called Geotools Geoserver GeoGig, a case study of use and utility field work. It was called GeoGit, now it's called GeoGig. Kind of same difference, just a new name. At the GIC, we have kind of a dual mission to provide, it's both academic and service oriented, and we provide opportunities for the students at Chico State to work in a GIS professional office and get experience that way. And we also provide GIS services to a variety of state agencies in terms of data development and services, as well as folks in our local region, so some cities and municipalities. And one of those clients that we work with is the Butte County Mosquito and Vector Control District. For those of you who aren't familiar with the duties of mosquito and vector control, their intention is to maintain and suppress mosquitoes and other vector nuisances in various ways, which I'm not incredibly familiar with because that's a whole different industry in science. The point here is that they have field crews that go out and they collect data, and they need a way to collect data and view data and edit that data in a better fashion than kind of what they were doing before, which was something that I'm sure as GIS people you've all come across where they're doing it in a paper setting. So additionally, that data that they collected needed to be versionable and they needed to have a history. So when they went to a source point of big pool of mosquitoes, they wanted to see what was going on there in the years prior. So it was important that they could see back to what had been done before and what the history of that site was. So we had our initial application that we built was on Esri and it worked and it works and they use it. But it obviously came at a cost. And that cost made it difficult to bring this sort of program to other districts in any industry really. So while that was always handy and worked out, it just comes at a cost. So we had an intention to find an alternate solution that provided the same functionality of the product that we had initially built for them. So we climbed a ladder to this guy and we looked at open source options. And this is kind of preaching to the choir, but obviously there's a lower cost of entry for open source. There is an initial time and therefore money commitment to finding stuff out, kind of a discovery process. But we knew that the tools that open source provided were commensurate and in some ways could exceed what the Esri stack provided for us. So there were additional benefits that were kind of for our shop. Since we were using the Esri ArcJS Mobile SDK, we were already extending that with some custom programming. So we were already making that time commitment to kind of creating a custom deal for our client. It allowed us to have a tool now that we could bring to other places without that cost. And in more of an open source sense, it allowed us to be involved in the community, to explore, to share what our findings were, to get involved, come to places like this and hear what people have to say and present our findings. So the proposed stack that we went with was Postgres, PostGIS extension with GeoServer, the GeoTools application on top of that and using GeoGig for kind of the versioning and the history. Since we already had that existing application basically pulled out, that data in shape files dumped it into the database connected to GeoServer. GeoServer allowed us to feed kind of base data which acted as the base map for the GeoTools application. Those three projects are all very mature, very stable. I don't think I need to go into a ton of detail on how those three components came up. And definitely on the GeoTools one because I was in the Java developer that did that. So that fell out. There's a lot of documentation on GeoTools development. So what we created was an application that could read all the same data that we were seeing before. They could query it. They could manipulate it. Zoom in, zoom out, pan, custom forms, all the things that they were able to do before. So where we're at is kind of right there, we have the application that's kind of doing that. We're stuck on that versioning part. That has proved to be more difficult than we anticipated. So when we wrote this abstract several months ago, it was like, yeah, we'll have that versioning bit done by then. But it's still kind of a thorn in the side. So some things worked and some things did not work. So in kind of typical open source fashion, you go back and look at what other tools are out there. So the database loading that works, feeding out data, that works. The application, like I mentioned, you can do all those things. But what I've seen here in the past couple days even is questioning that use of GeoGig whether in this particular instance, if it is providing almost too much complexity for the application that we need it for. So in an interim, even before we got to this conference here, we were kind of doing this bastardized version of various states of shapefile. Our client was not a complex enough use case to where we really needed to go into this real deep versioned editing. There's only 15 or so guys in the field. They were never overlapping areas. So the check and check out process was not as complex as it needed to be. But that's not always going to be the case for other utility agencies or the like. So I've seen now several uses that talks have gone through the past couple days of people using the GitHub IOPages with the GeoJSON support. That seems like a real viable option. There may be a lack of understanding on my part about why that GeoGig platform needs to exist when the GitHub platform is kind of doing the same thing kind of all on its own. If anybody has that knowledge, I'd love to talk to you guys about it. But given that this kind of newer solution is, you know, we haven't really worked with it, I have an idea of how it's going to work in my head. And I'll, you know, work on that when I get back to the office. But if you guys are looking back on videos, check out Landon Reed's discussion. He's a guy that works for the Atlanta Regional Commission. And they were doing, using this for versioning and kind of in a way kind of like a crowdsourcing of data editing for roads in the Atlanta region. So that's a real complex data set. A lot of features, a lot of attributes, and allowed for people to pull and push data and manipulate it and let it be reconciled by, you know, the administrators. So in this case, where we would use this is probably getting that Geo, pulling that GeoJSON down, pushing it into GeoTools, manipulating it there and then pushing it back up for it to be reconciled with kind of the master database. So there's future work to be done on the app, obviously. And one of those things is to incorporate that technique and see how it goes and report back to the community and what we found out. We'd also like to improve the UI and the SLDs. I don't know how many folks here have worked with SLDs, but they are kind of a slog and they're a bit difficult to work with. So improving that cartography is something that's very important for the field crew. You can incorporate a GPS dongle onto the field laptops that they're using so they can, you know, utilize that when they're out in the field rather than having to pick a point on the map. And then, of course, we'd need to test it and debug it and test it and debug it and get that feedback and really flesh out that product for them. And for us as a shop and being involved in the open source community, you know, we have a responsibility to become more involved. And I've been kind of dabbling in various facets of open source GIS for several years. And coming to this conference, I mean, you see a lot of the names you recognize, but then there's a whole host of other people that are doing amazing things. And they're putting things out there. So we have a need to go and discover more. And there's a ton of stuff on Twitter and there's a ton of blogs. Obviously, a lot of stuff is being shared on GitHub. So I've met also a lot of people that are not really, they're really noobs in this kind of open source GIS. So there's a ton of stuff out there. So go and find it. Go and look at it. And then contribute more. Put the stuff out there that we're finding, even if it doesn't work. So GeoGig didn't work like we wanted to. Maybe this can help some of the developers to streamline their efforts and getting more folks like us to be able to use it. So I would do have to mention that. What kind of held us back, I feel like, was the integration into GeoServer, the documentation on that was not as much as we needed. And then a thing that I've seen stressed on a lot this week is the need for additional collaboration locally and on the web. So getting involved with folks in our community that are not just GIS people, but in different industries that have interest in spatial and kind of really maximizing everybody's efforts and interests into something that's going to work for everybody. So yeah, that's what I've got. So if there's any questions, we'd be glad to take them. Hello. I was talking to somebody regarding the use of triggers as a way to provide version support for multiple editors. Have you looked at that at all? No, because for this case, the lack of overlap, we didn't need that level of complexity, but look into that. One of the improvements you wanted to make was around SLDs and kind of the difficulty in working with them. Do you have any ideas or plans as to how to improve that? Well, yeah, it's a better editor. The composer that was an open GioSuite was nice. I believe there's a QGIS plug-in. One of the most successful ones I was able to use was in Udig. Actually, you could export out the SLD or export out the XML and create an SLD. But yeah, there was a lot of stuff that was case-sensitive and it would just throw errors and GeoTools. It was just trial and error. You just look at the log and see what was making it break and go back and fix it. But there was a company that made, I believe it was a German company, you could make your SLDs and ARC and spit them out. That worked kind of. But once you got into any sort of regreel deep symbology, like kind of thematic stuff, it would just kind of bit the dust. There's a lot of stuff happening with CSS, cartography with CSS. I feel like that's where a lot of the effort is at. I'm curious what will happen with SLDs and their use. I think they're still widely used. Yeah, actually, I was just going to comment on that. There is a CSS plug-in to GeoServer. I don't know if you've looked at that, but we use that and like you say for thematic stuff, it's way, way nicer. It does generate the SLD behind the scenes, so we would definitely recommend looking at that. What's that called? It's just a whatever, CSS plug-in to GeoServer. It's easy to install and definitely a lot nicer. And then just one other thing about using GitHub and why GeoGig, I'm not an expert on either of them, but I kind of get the impression that the GitHub stuff is for relatively small files. Yeah. Maybe a few thousand records or something. We deal with much larger datasets and I think GeoGig is maybe aiming to work with larger datasets. Yeah, and that's something that wasn't, I think, a lot of the time. That wasn't addressed fully in either of those talks that I'd seen, actually. Yeah, it will work for the smaller stuff. And I think for the larger ones, they were kind of breaking them down into kind of more regional deals. But yeah, and I think that's kind of a limitation for GeoJ, so anyway it's kind of got, it's slim but it's still got a pretty decent overhead. But for simpler cases, yeah, I think it's just like in Vladimir's talk about simplifying, you know, if you don't need to go with, I'm hesitant to call it bloat, but sometimes you don't need the solution that does everything. You need the one that does what you're working on. Thank you. Thank you.
 All right, I think it's about 10 o'clock, so let's get the show on the road.  My name is Peter Hansen.  I work for the Geographical Information Center at California State University, Chico.  I have a presentation called Geotools Geoserver GeoGig, a case study of use and utility field  work.  It was called GeoGit, now it's called GeoGig.  Kind of same difference, just a new name.  At the GIC, we have kind of a dual mission to provide, it's both academic and service  oriented, and we provide opportunities for the students at Chico State to work in a GIS  professional office and get experience that way.  And we also provide GIS services to a variety of state agencies in terms of data development  and services, as well as folks in our local region, so some cities and municipalities.  And one of those clients that we work with is the Butte County Mosquito and Vector Control  District.  For those of you who aren't familiar with the duties of mosquito and vector control,  their intention is to maintain and suppress mosquitoes and other vector nuisances in various  ways, which I'm not incredibly familiar with because that's a whole different industry  in science.  The point here is that they have field crews that go out and they collect data, and they  need a way to collect data and view data and edit that data in a better fashion than kind  of what they were doing before, which was something that I'm sure as GIS people you've  all come across where they're doing it in a paper setting.  So additionally, that data that they collected needed to be versionable and they needed to  have a history.  So when they went to a source point of big pool of mosquitoes, they wanted to see what  was going on there in the years prior.  So it was important that they could see back to what had been done before and what the  history of that site was.  So we had our initial application that we built was on Esri and it worked and it works  and they use it.  But it obviously came at a cost.  And that cost made it difficult to bring this sort of program to other districts in any  industry really.  So while that was always handy and worked out, it just comes at a cost.  So we had an intention to find an alternate solution that provided the same functionality  of the product that we had initially built for them.  So we climbed a ladder to this guy and we looked at open source options.  And this is kind of preaching to the choir, but obviously there's a lower cost of entry  for open source.  There is an initial time and therefore money commitment to finding stuff out, kind of a  discovery process.  But we knew that the tools that open source provided were commensurate and in some ways  could exceed what the Esri stack provided for us.  So there were additional benefits that were kind of for our shop.  Since we were using the Esri ArcJS Mobile SDK, we were already extending that with some custom  programming.  So we were already making that time commitment to kind of creating a custom deal for our  client.  It allowed us to have a tool now that we could bring to other places without that cost.  And in more of an open source sense, it allowed us to be involved in the community, to explore,  to share what our findings were, to get involved, come to places like this and hear what people  have to say and present our findings.  So the proposed stack that we went with was Postgres, PostGIS extension with GeoServer,  the GeoTools application on top of that and using GeoGig for kind of the versioning and  the history.  Since we already had that existing application basically pulled out, that data in shape files  dumped it into the database connected to GeoServer.  GeoServer allowed us to feed kind of base data which acted as the base map for the GeoTools  application.  Those three projects are all very mature, very stable.  I don't think I need to go into a ton of detail on how those three components came up.  And definitely on the GeoTools one because I was in the Java developer that did that.  So that fell out.  There's a lot of documentation on GeoTools development.  So what we created was an application that could read all the same data that we were  seeing before.  They could query it.  They could manipulate it.  Zoom in, zoom out, pan, custom forms, all the things that they were able to do before.  So where we're at is kind of right there, we have the application that's kind of doing  that.  We're stuck on that versioning part.  That has proved to be more difficult than we anticipated.  So when we wrote this abstract several months ago, it was like, yeah, we'll have that versioning  bit done by then.  But it's still kind of a thorn in the side.  So some things worked and some things did not work.  So in kind of typical open source fashion, you go back and look at what other tools are  out there.  So the database loading that works, feeding out data, that works.  The application, like I mentioned, you can do all those things.  But what I've seen here in the past couple days even is questioning that use of GeoGig  whether in this particular instance, if it is providing almost too much complexity for  the application that we need it for.  So in an interim, even before we got to this conference here, we were kind of doing this  bastardized version of various states of shapefile.  Our client was not a complex enough use case to where we really needed to go into this  real deep versioned editing.  There's only 15 or so guys in the field.  They were never overlapping areas.  So the check and check out process was not as complex as it needed to be.  But that's not always going to be the case for other utility agencies or the like.  So I've seen now several uses that talks have gone through the past couple days of people  using the GitHub IOPages with the GeoJSON support.  That seems like a real viable option.  There may be a lack of understanding on my part about why that GeoGig platform needs  to exist when the GitHub platform is kind of doing the same thing kind of all on its  own.  If anybody has that knowledge, I'd love to talk to you guys about it.  But given that this kind of newer solution is, you know, we haven't really worked with  it, I have an idea of how it's going to work in my head.  And I'll, you know, work on that when I get back to the office.  But if you guys are looking back on videos, check out Landon Reed's discussion.  He's a guy that works for the Atlanta Regional Commission.  And they were doing, using this for versioning and kind of in a way kind of like a crowdsourcing  of data editing for roads in the Atlanta region.  So that's a real complex data set.  A lot of features, a lot of attributes, and allowed for people to pull and push data and  manipulate it and let it be reconciled by, you know, the administrators.  So in this case, where we would use this is probably getting that Geo, pulling that GeoJSON  down, pushing it into GeoTools, manipulating it there and then pushing it back up for it  to be reconciled with kind of the master database.  So there's future work to be done on the app, obviously.  And one of those things is to incorporate that technique and see how it goes and report  back to the community and what we found out.  We'd also like to improve the UI and the SLDs.  I don't know how many folks here have worked with SLDs, but they are kind of a slog and  they're a bit difficult to work with.  So improving that cartography is something that's very important for the field crew.  You can incorporate a GPS dongle onto the field laptops that they're using so they can,  you know, utilize that when they're out in the field rather than having to pick a point  on the map.  And then, of course, we'd need to test it and debug it and test it and debug it and  get that feedback and really flesh out that product for them.  And for us as a shop and being involved in the open source community, you know, we have  a responsibility to become more involved.  And I've been kind of dabbling in various facets of open source GIS for several years.  And coming to this conference, I mean, you see a lot of the names you recognize, but  then there's a whole host of other people that are doing amazing things.  And they're putting things out there.  So we have a need to go and discover more.  And there's a ton of stuff on Twitter and there's a ton of blogs.  Obviously, a lot of stuff is being shared on GitHub.  So I've met also a lot of people that are not really, they're really noobs in this kind  of open source GIS.  So there's a ton of stuff out there.  So go and find it.  Go and look at it.  And then contribute more.  Put the stuff out there that we're finding, even if it doesn't work.  So GeoGig didn't work like we wanted to.  Maybe this can help some of the developers to streamline their efforts and getting more  folks like us to be able to use it.  So I would do have to mention that.  What kind of held us back, I feel like, was the integration into GeoServer, the documentation  on that was not as much as we needed.  And then a thing that I've seen stressed on a lot this week is the need for additional  collaboration locally and on the web.  So getting involved with folks in our community that are not just GIS people, but in different  industries that have interest in spatial and kind of really maximizing everybody's efforts  and interests into something that's going to work for everybody.  So yeah, that's what I've got.  So if there's any questions, we'd be glad to take them.  Hello.  I was talking to somebody regarding the use of triggers as a way to provide version support  for multiple editors.  Have you looked at that at all?  No, because for this case, the lack of overlap, we didn't need that level of complexity, but  look into that.  One of the improvements you wanted to make was around SLDs and kind of the difficulty  in working with them.  Do you have any ideas or plans as to how to improve that?  Well, yeah, it's a better editor.  The composer that was an open GioSuite was nice.  I believe there's a QGIS plug-in.  One of the most successful ones I was able to use was in Udig.  Actually, you could export out the SLD or export out the XML and create an SLD.  But yeah, there was a lot of stuff that was case-sensitive and it would just throw errors  and GeoTools.  It was just trial and error.  You just look at the log and see what was making it break and go back and fix it.  But there was a company that made, I believe it was a German company, you could make your  SLDs and ARC and spit them out.  That worked kind of.  But once you got into any sort of regreel deep symbology, like kind of thematic stuff,  it would just kind of bit the dust.  There's a lot of stuff happening with CSS, cartography with CSS.  I feel like that's where a lot of the effort is at.  I'm curious what will happen with SLDs and their use.  I think they're still widely used.  Yeah, actually, I was just going to comment on that.  There is a CSS plug-in to GeoServer.  I don't know if you've looked at that, but we use that and like you say for thematic  stuff, it's way, way nicer.  It does generate the SLD behind the scenes, so we would definitely recommend looking at  that.  What's that called?  It's just a whatever, CSS plug-in to GeoServer.  It's easy to install and definitely a lot nicer.  And then just one other thing about using GitHub and why GeoGig, I'm not an expert on  either of them, but I kind of get the impression that the GitHub stuff is for relatively small  files.  Yeah.  Maybe a few thousand records or something.  We deal with much larger datasets and I think GeoGig is maybe aiming to work with larger  datasets.  Yeah, and that's something that wasn't, I think, a lot of the time.  That wasn't addressed fully in either of those talks that I'd seen, actually.  Yeah, it will work for the smaller stuff.  And I think for the larger ones, they were kind of breaking them down into kind of more  regional deals.  But yeah, and I think that's kind of a limitation for GeoJ, so anyway it's kind of got, it's  slim but it's still got a pretty decent overhead.  But for simpler cases, yeah, I think it's just like in Vladimir's talk about simplifying,  you know, if you don't need to go with, I'm hesitant to call it bloat, but sometimes you  don't need the solution that does everything.  You need the one that does what you're working on.  Thank you.  Thank you.
01_Peter_Hansen_720p_0_107.png 01_Peter_Hansen_720p_108_1799.png 01_Peter_Hansen_720p_1800_4103.png 01_Peter_Hansen_720p_4104_4487.png 01_Peter_Hansen_720p_4488_5075.png 01_Peter_Hansen_720p_5076_5519.png 01_Peter_Hansen_720p_5520_6455.png 01_Peter_Hansen_720p_6456_7379.png 01_Peter_Hansen_720p_7380_7607.png 01_Peter_Hansen_720p_7608_7763.png 01_Peter_Hansen_720p_7764_8051.png 01_Peter_Hansen_720p_8052_10319.png 01_Peter_Hansen_720p_10320_11027.png 01_Peter_Hansen_720p_11028_11171.png 01_Peter_Hansen_720p_11172_11675.png 01_Peter_Hansen_720p_11676_16511.png 01_Peter_Hansen_720p_16512_18227.png 01_Peter_Hansen_720p_18228_21059.png 01_Peter_Hansen_720p_21060_22307.png 01_Peter_Hansen_720p_22308_22379.png 01_Peter_Hansen_720p_22380_22763.png 01_Peter_Hansen_720p_22764_23843.png 01_Peter_Hansen_720p_23844_25067.png 01_Peter_Hansen_720p_25068_29018.png