Geomajas: Where did we go wrong?
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
May, Oliver
Geomajas is an open source geospatial web development framework that has been around for almost a decade. Despite lots of hard work from our community, until today we have not been able to turn Geomajas into the big success we initially planned. I have been around Geomajas from almost the beginning, as developer, architect and PSC member. During this talk I will shine a personal light on my hate love affair with Geomajas. I will be sharing my observations on some decisions that had a huge effect on the evolution of the framework and try to pass on my vision on how to learn from both our mistakes and successes. To conclude I will to reach out to fellow OSGeo projects, and share my view on how we, as the open source geospatial community, can help each other in becoming more successful open source projects.
 Thank you. Thank you all for coming. So I'm Oliver May. I'm part of the GeoMaias project steering committee. And I'm going to present today some slides about the stuff that I encountered the last ten years working on the GeoMaias project. So about me, my hobby is procrastination. I invent weird presentation titles. And then eight days before a conference, this little guy gets born. So sorry if my presentation this year is a little bit sloppy, but I had some slight sleep deprivation. So anyway, like I already said in the introduction, I'm going to talk about some experience that I had the last years working on the GeoMaias project. For the people that don't know what GeoMaias exactly is, it's an open source platform to create web, I'm getting a lot of feedback here from the microphone, for creating web GIS applications. So it's not really an application by itself, but more development framework written in Java. It contains a powerful server that allows you to read and aggregate various GIS data sources and process them to be sent and presented in the browser. It has built-in security, reprojection, and all other sorts of processing possibilities. And it's also possible to apply domain logic. Of course. Better? Yeah. All right. Now I got feedback again. So it contains a powerful server. I already told that. It is possible to apply your domain logic to your different data layers that you configure in the server. Representation in the web browser happens through web clients where you can display and edit your data in the browser. Like I said, that's quite important. It's a framework. It's not an application, and it's written in Java. This is really annoying. Yeah, I'll do that. All right. So it's a framework, not an application. It's written in Java. There is a geomaias position. If we look at quadrants and define two axes, where the horizontal axis is the amount of individual users that will use an application built or will use an application. And vertically, you say the amount of data layers that you want to visualize or process at one time. Typically, desktop applications, such as QGIS or SD Desktop, go the whole way from less layers to a lot of layers. But it's typically a single user that is using one application. If you want to outreach your application to a lot of users to a big audience, and it's limited to up to like five different data layers, typically this is where Cartodibé, and I won't correct myself, comes above or Google Apps. Geomaias is more located at the top where we try to enable it to show a lot of data layers to a lot of different users. For example, we have the subsoil project in Flanders, which consists of more than 200 data layers. Of course, they're not all displayed at the same time. But that's one example. So Geomaias started off in 2006 when the Flemish government was looking for an open source generic web GIS application. In 2007 and 2008, the first prototype of this application was built, and the first lines of code were written. In 2010, we graduated as an OSGEO project, and international projects and applications began to start. So during the rest of 2010, we continued worldwide exposure and did projects in Australia, Sweden, and so on. And one of the major milestones was in 2013, the birth of the GW2 client, which I'll come back to a little bit later. And also very important, the last starting in 2015, and this year, we are also outreaching and talking together with other open source GIS communities, which is also what we want to do more and more in the future, and OSGEO is an ideal platform for that. So today, I'm going to touch a couple of subjects that during the time in this open source project had quite an impact on the evolution of the project. The first one is funding. Of course, without funding, there is no open source project, or there is no project. So for the funding of GeoMaias, we actually have to rely on three things. The first one is the community. An open source project like GeoMaias is carried by a lot of volunteers who do the extra mile because it's open source, because it's their project. I've noticed that it's not always very easy to motivate people like Arnov also said in the opening session earlier today. It's not easy to manage a community. It's not like a team of staff that you can say, I won't pay you when you don't do your job. That's open source. It's quite difficult. Good events like this week are a good way of getting return from the things you do in an open source project. Now the second thing is that we have GeoSpark, which is an organization or a company that provides commercial services to GeoMaias. They actually build revenue on top of GeoMaias, for example, service level agreements or dual licensing. These revenues flow back into the GeoMaias project, which is actually quite a huge part of the funding. The third way is by users or projects that use GeoMaias, they fix or improve things and we want them to flow back into the open source projects. It's a bit, they're also part of the community, but the big difference is that the people who work on projects, they're actually paid to commit to the project. Now the problem with this kind of funding is mostly focused. Projects or applications that use open source frameworks are focused on getting their application up and running and building their thing. The choice to use open source is mostly, hey, it's free, let's use it and that's it. The main challenge is to convince these companies that by using open source software, they actually become part of open source. The open source projects and thus eventually themselves will become better if they commit if they return. But it's very, very difficult to tell that to other companies. Another case that I want to discuss today is technology. Somehow this has been the Achilles heel from the GeoMaias project for quite some while. The reason for this is mostly because the web evolves very, very, very fast. You may say that it goes too fast. So in the last decade, the web has changed from a robust contract-first XML-based thing to a collection of small services that are simple and stupid. Others talk to these services, these services talk to each other through very basic HTTP protocol, web hooks, REST services and so on. Also a lot of the processing to present information through the web has actually shifted from servers which were portals in the years 2000 to browsers, apps, running on computers, phones and you name it. So this shift has also caused a revolution in the web world of which HTML5 is one of the important results. So the web is really evolving very, very fast. So back to GeoMaias, until today GeoMaias had three different web clients, each kind of obsoleting the last one, not really by design but more by limited features. As long as there is funding for one of these clients, we can keep continue investing in the client but when the funding runs out, it becomes obsolete. So in 2006, at the whole beginning of the GeoMaias project, this is also quite the early ages of the web in terms of where we stand now. The down of, yeah, maybe not the down but web 2.0 was around that time. So we started looking at available libraries that we could use for our clients. Dojo at that time was a good candidate. I provided a lot of features and widgets out of the box, such as toolbars, tables and you name it. So based on this Dojo toolkit, the GeoMaias Dojo client was built. This client was focused on bringing desktop GIS to the web, which was at that time our mindset. I kind of came back from it today because desktop GIS is something that you should keep on desktop GIS and web interfaces should be as simple as possible, more like apps on your phone, very simple. So it had a huge amount of GIS related widgets, such as attributes, feature table, editing, searching and so on. So at that time in 2006, using JavaScript, which Dojo was, was just a JavaScript framework, without any further abstraction, proved to be a real struggle. The reason for that was that browsers at that time were as incompatible as can be. You would get everything correctly on one browser and then Microsoft pushed a new browser and it was all fucked up again and then companies wanted to stay at Internet Explorer 6 for some obscure reason. So this was really not a good time to be in this JavaScript world. Also because we had a real tight integration with the API of Dojo, upgrading to new Dojo versions was also really getting, being in struggle. We almost were putting more efforts in getting our libraries up to date than to actually introducing new features. So we actually stopped at a certain version of Dojo. So not able to update this Dojo version also meant that look and feel became outdated really fast. So in 2010, we went looking for an alternative. At that point, there was a big fuss about the Google Web toolkit, which forms an abstraction between JavaScript and your programming language, which is for GWT, is Java. The only problem was we didn't have any widgets in Google Web toolkit. So to solve that, we decided to use the smart GWT project, which is actually a wrapper around smart client, which is also a JavaScript library like Dojo. So this GeoMaias client also, like the Dojo client, contained a high amount of GIS-related widgets and features comparable to the Dojo client. So the fact that it's a wrapper around smart client also means that it's slow in development. We had to wait for smart clients to add additional features, and then we had to wait for the smart GWT project to release a new version and so on. So we were missing two, three, four months on the development cycle of the web, which was at that point going really, really fast. Also the compilation and development of this smart GWT application was quite slow. Underly, it had compiled JavaScript code, so once you push your debugger deeper and deeper, at a certain moment you hit this compiled JavaScript and you're just completely lost. So again, some years later in 2013, we actually started the GWT2 client, which is based on the Google Web Toolkit, but we decided that we didn't want to bring extra widgets and feature list tables and all that stuff, because most of the time in projects when applications are implemented, they write those things themselves or they can look for their own library. So it's far more lightweight than its predecessors. Also able to run on mobile hardware without any problem, which that was also very convenient at that time. So we built a minimum set of widgets developed from scratch and experience, yeah, that I already said that. So again, we had some drawbacks. All of a sudden Google decided to abandon the Google Web Toolkit. They didn't really abandon it, but what they did is let's put it on GitHub or wherever and let's create, let it be an open source project on their own. But of course, funding was cut off because Google itself wasn't investing anymore. Another thing about GWT is that the developers that know GWT are scarce and the learning curve is quite steep. And in the meantime, with HTML5 and browsers beginning to behave to the standards, frameworks like GWT were actually more complex than the underlying technology that they were trying to abstract being HTML5. So that's actually the point where we're today. So where do we want to go from here with all this information and these three clients and all the heavy weight? We're going to strip the functionality of our client more down and focus on a simple REST API because we also learn from other projects in the JavaScript world that web developers really like just to call APIs and write all the REST themselves with technologies of their choice. So that's actually where we're going from now with the client. So I gave two cases of things that went not so well in the last years. And what they really challenge is the sustainability of a project. We have a good project. We are well organized. We are known international. Maybe we want to have a broader user range, but yeah, you can't do a lot about it besides coming here and speaking about it. So this is actually the challenge that we have for the following years and it counts for every open source project. We need to keep up with the evolution of the web and also make sure that we don't lose funding so that we can continue. And I think that OSGEO and everybody that's here this week are part of the community that is necessary to keep this open source project going and running. So that concludes my presentation. Thank you for listening. Thank you for the opportunity to put you on a mess at the start. So you just applauded Oliver. Is there anyone about any questions they want to ask him about the GioMahas experience specifically or about the more general experience of changing technology? So in hindsight, would you say perhaps that you've tied yourself too much to frameworks or it's easy to set inside but perhaps 210 could have just done everything yourself? JavaScript not tied yourself to framework with any wall step on the side? So the learning? Yeah, but the thing is if you use an existing framework, you win a lot on a short time. And as a matter of fact, for our server component, we use Spring and we use Hibernate and we depend on GioTools and that works perfectly. It's only this web thing. The web just moves so fast that it makes it very hard. If you stop development for four months, you're already behind and almost impossible to catch up. But how would you tie yourself to frameworks with link button as? Well that's... Types. Yeah. And somehow it got out of my slide. But that's one of the things that we want to do now, provide an API and provide plugins for leaflet and open layers so that they can communicate with GioMahas. And then we don't have to worry about the client anymore and more if we have resources to spare, we can even help out open layers and leaflets. So yeah. Any other questions for Oliver? Second round of applause.
 Thank you.  Thank you all for coming.  So I'm Oliver May.  I'm part of the GeoMaias project steering committee.  And I'm going to present today some slides about the stuff that I encountered the last  ten years working on the GeoMaias project.  So about me, my hobby is procrastination.  I invent weird presentation titles.  And then eight days before a conference, this little guy gets born.  So sorry if my presentation this year is a little bit sloppy, but I had some slight sleep  deprivation.  So anyway, like I already said in the introduction, I'm going to talk about some experience that  I had the last years working on the GeoMaias project.  For the people that don't know what GeoMaias exactly is, it's an open source platform to  create web, I'm getting a lot of feedback here from the microphone, for creating web  GIS applications.  So it's not really an application by itself, but more development framework written in  Java.  It contains a powerful server that allows you to read and aggregate various GIS data  sources and process them to be sent and presented in the browser.  It has built-in security, reprojection, and all other sorts of processing possibilities.  And it's also possible to apply domain logic.  Of course.  Better?  Yeah.  All right.  Now I got feedback again.  So it contains a powerful server.  I already told that.  It is possible to apply your domain logic to your different data layers that you configure  in the server.  Representation in the web browser happens through web clients where you can display and edit  your data in the browser.  Like I said, that's quite important.  It's a framework.  It's not an application, and it's written in Java.  This is really annoying.  Yeah, I'll do that.  All right.  So it's a framework, not an application.  It's written in Java.  There is a geomaias position.  If we look at quadrants and define two axes, where the horizontal axis is the amount of  individual users that will use an application built or will use an application.  And vertically, you say the amount of data layers that you want to visualize or process  at one time.  Typically, desktop applications, such as QGIS or SD Desktop, go the whole way from less  layers to a lot of layers.  But it's typically a single user that is using one application.  If you want to outreach your application to a lot of users to a big audience, and it's  limited to up to like five different data layers, typically this is where Cartodibé,  and I won't correct myself, comes above or Google Apps.  Geomaias is more located at the top where we try to enable it to show a lot of data  layers to a lot of different users.  For example, we have the subsoil project in Flanders, which consists of more than 200  data layers.  Of course, they're not all displayed at the same time.  But that's one example.  So Geomaias started off in 2006 when the Flemish government was looking for an open source  generic web GIS application.  In 2007 and 2008, the first prototype of this application was built, and the first lines  of code were written.  In 2010, we graduated as an OSGEO project, and international projects and applications  began to start.  So during the rest of 2010, we continued worldwide exposure and did projects in Australia, Sweden,  and so on.  And one of the major milestones was in 2013, the birth of the GW2 client, which I'll come  back to a little bit later.  And also very important, the last starting in 2015, and this year, we are also outreaching  and talking together with other open source GIS communities, which is also what we want  to do more and more in the future, and OSGEO is an ideal platform for that.  So today, I'm going to touch a couple of subjects that during the time in this open source project  had quite an impact on the evolution of the project.  The first one is funding.  Of course, without funding, there is no open source project, or there is no project.  So for the funding of GeoMaias, we actually have to rely on three things.  The first one is the community.  An open source project like GeoMaias is carried by a lot of volunteers who do the extra mile  because it's open source, because it's their project.  I've noticed that it's not always very easy to motivate people like Arnov also said in  the opening session earlier today.  It's not easy to manage a community.  It's not like a team of staff that you can say, I won't pay you when you don't do your  job.  That's open source.  It's quite difficult.  Good events like this week are a good way of getting return from the things you do in  an open source project.  Now the second thing is that we have GeoSpark, which is an organization or a company that  provides commercial services to GeoMaias.  They actually build revenue on top of GeoMaias, for example, service level agreements or dual  licensing.  These revenues flow back into the GeoMaias project, which is actually quite a huge part  of the funding.  The third way is by users or projects that use GeoMaias, they fix or improve things and  we want them to flow back into the open source projects.  It's a bit, they're also part of the community, but the big difference is that the people  who work on projects, they're actually paid to commit to the project.  Now the problem with this kind of funding is mostly focused.  Projects or applications that use open source frameworks are focused on getting their application  up and running and building their thing.  The choice to use open source is mostly, hey, it's free, let's use it and that's it.  The main challenge is to convince these companies that by using open source software, they actually  become part of open source.  The open source projects and thus eventually themselves will become better if they commit  if they return.  But it's very, very difficult to tell that to other companies.  Another case that I want to discuss today is technology.  Somehow this has been the Achilles heel from the GeoMaias project for quite some while.  The reason for this is mostly because the web evolves very, very, very fast.  You may say that it goes too fast.  So in the last decade, the web has changed from a robust contract-first XML-based thing  to a collection of small services that are simple and stupid.  Others talk to these services, these services talk to each other through very basic HTTP  protocol, web hooks, REST services and so on.  Also a lot of the processing to present information through the web has actually shifted from  servers which were portals in the years 2000 to browsers, apps, running on computers, phones  and you name it.  So this shift has also caused a revolution in the web world of which HTML5 is one of  the important results.  So the web is really evolving very, very fast.  So back to GeoMaias, until today GeoMaias had three different web clients, each kind  of obsoleting the last one, not really by design but more by limited features.  As long as there is funding for one of these clients, we can keep continue investing in  the client but when the funding runs out, it becomes obsolete.  So in 2006, at the whole beginning of the GeoMaias project, this is also quite the early  ages of the web in terms of where we stand now.  The down of, yeah, maybe not the down but web 2.0 was around that time.  So we started looking at available libraries that we could use for our clients.  Dojo at that time was a good candidate.  I provided a lot of features and widgets out of the box, such as toolbars, tables and you  name it.  So based on this Dojo toolkit, the GeoMaias Dojo client was built.  This client was focused on bringing desktop GIS to the web, which was at that time our  mindset.  I kind of came back from it today because desktop GIS is something that you should keep  on desktop GIS and web interfaces should be as simple as possible, more like apps on your  phone, very simple.  So it had a huge amount of GIS related widgets, such as attributes, feature table, editing,  searching and so on.  So at that time in 2006, using JavaScript, which Dojo was, was just a JavaScript framework,  without any further abstraction, proved to be a real struggle.  The reason for that was that browsers at that time were as incompatible as can be.  You would get everything correctly on one browser and then Microsoft pushed a new browser  and it was all fucked up again and then companies wanted to stay at Internet Explorer 6 for  some obscure reason.  So this was really not a good time to be in this JavaScript world.  Also because we had a real tight integration with the API of Dojo, upgrading to new Dojo  versions was also really getting, being in struggle.  We almost were putting more efforts in getting our libraries up to date than to actually  introducing new features.  So we actually stopped at a certain version of Dojo.  So not able to update this Dojo version also meant that look and feel became outdated really  fast.  So in 2010, we went looking for an alternative.  At that point, there was a big fuss about the Google Web toolkit, which forms an abstraction  between JavaScript and your programming language, which is for GWT, is Java.  The only problem was we didn't have any widgets in Google Web toolkit.  So to solve that, we decided to use the smart GWT project, which is actually a wrapper around  smart client, which is also a JavaScript library like Dojo.  So this GeoMaias client also, like the Dojo client, contained a high amount of GIS-related  widgets and features comparable to the Dojo client.  So the fact that it's a wrapper around smart client also means that it's slow in development.  We had to wait for smart clients to add additional features, and then we had to wait for the  smart GWT project to release a new version and so on.  So we were missing two, three, four months on the development cycle of the web, which  was at that point going really, really fast.  Also the compilation and development of this smart GWT application was quite slow.  Underly, it had compiled JavaScript code, so once you push your debugger deeper and  deeper, at a certain moment you hit this compiled JavaScript and you're just completely lost.  So again, some years later in 2013, we actually started the GWT2 client, which is based on  the Google Web Toolkit, but we decided that we didn't want to bring extra widgets and  feature list tables and all that stuff, because most of the time in projects when applications  are implemented, they write those things themselves or they can look for their own library.  So it's far more lightweight than its predecessors.  Also able to run on mobile hardware without any problem, which that was also very convenient  at that time.  So we built a minimum set of widgets developed from scratch and experience, yeah, that I  already said that.  So again, we had some drawbacks.  All of a sudden Google decided to abandon the Google Web Toolkit.  They didn't really abandon it, but what they did is let's put it on GitHub or wherever  and let's create, let it be an open source project on their own.  But of course, funding was cut off because Google itself wasn't investing anymore.  Another thing about GWT is that the developers that know GWT are scarce and the learning  curve is quite steep.  And in the meantime, with HTML5 and browsers beginning to behave to the standards, frameworks  like GWT were actually more complex than the underlying technology that they were trying  to abstract being HTML5.  So that's actually the point where we're today.  So where do we want to go from here with all this information and these three clients and  all the heavy weight?  We're going to strip the functionality of our client more down and focus on a simple  REST API because we also learn from other projects in the JavaScript world that web  developers really like just to call APIs and write all the REST themselves with technologies  of their choice.  So that's actually where we're going from now with the client.  So I gave two cases of things that went not so well in the last years.  And what they really challenge is the sustainability of a project.  We have a good project.  We are well organized.  We are known international.  Maybe we want to have a broader user range, but yeah, you can't do a lot about it besides  coming here and speaking about it.  So this is actually the challenge that we have for the following years and it counts  for every open source project.  We need to keep up with the evolution of the web and also make sure that we don't lose  funding so that we can continue.  And I think that OSGEO and everybody that's here this week are part of the community that  is necessary to keep this open source project going and running.  So that concludes my presentation.  Thank you for listening.  Thank you for the opportunity to put you on a mess at the start.  So you just applauded Oliver.  Is there anyone about any questions they want to ask him about the GioMahas experience  specifically or about the more general experience of changing technology?  So in hindsight, would you say perhaps that you've tied yourself too much to frameworks  or it's easy to set inside but perhaps 210 could have just done everything yourself?  JavaScript not tied yourself to framework with any wall step on the side?  So the learning?  Yeah, but the thing is if you use an existing framework, you win a lot on a short time.  And as a matter of fact, for our server component, we use Spring and we use Hibernate and we  depend on GioTools and that works perfectly.  It's only this web thing.  The web just moves so fast that it makes it very hard.  If you stop development for four months, you're already behind and almost impossible to catch  up.  But how would you tie yourself to frameworks with link button as?  Well that's...  Types.  Yeah.  And somehow it got out of my slide.  But that's one of the things that we want to do now, provide an API and provide plugins  for leaflet and open layers so that they can communicate with GioMahas.  And then we don't have to worry about the client anymore and more if we have resources  to spare, we can even help out open layers and leaflets.  So yeah.  Any other questions for Oliver?  Second round of applause.
foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_0_659.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_660_707.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_708_827.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_828_1631.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_1632_2051.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_2052_5471.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_5472_5987.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_5988_7787.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_7788_8507.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_8508_8879.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_8880_14759.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_14760_16151.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_16152_16775.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_16776_18443.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_18444_19391.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_19392_19487.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_19488_20795.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_20796_21683.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_21684_21803.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_21804_23879.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_23880_24023.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_24024_26783.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_26784_28787.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_28788_31223.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_31224_31415.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_31416_33143.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_33144_33191.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_33192_33299.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_33300_33395.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_33396_35291.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_35292_36107.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_36108_36191.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_36192_36251.png foss4g-2016-1294-geomajas_where_did_we_go_wrong-hd_36252_36350.png