Water Quality Forecast And Information System Built With FLOSS Software
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Craciunescu, Vasile (Romanian National Meteorological Administration)
Water quality is a major problem nowadays around the world. CLEANWATER system combine various information and complex data in order to evaluate the present level of nutrient pollution in vulnerable areas, as well as to assess the cost-efficiency of the measures that could be applied. Through a simple and intuitive web interface, CLEANWATER offers the decision makers a spatial aware tool to (1) create scenarios related to the human activities and climate changes, (2) send those scenarios to numerical models to model future evolution of water quality and (3) view, query and perform spatial analysis of the simulation results. The system was implemented in a test river basin (Barlad River Basin in Eastern part of Romanian) and started to contribute to the development of a modern water management system, according to EU legislation (e.g. Water Framework Directive, Nitrates Directive). The future plan is to replicate the system at national and international level. The system is build entirely with standard compliant free and open source software applications like OpenLayers, ExtJS, PostGIS, GeoServer and GDAL.
 My name is Vasile Krachunosko, I'm from Romania, from Bucharest, from the National Meteorological Administration, Remote Sensing and GIS department. I'm also leading the OSG or Romanian group and I'm always scared in presenting in Phosphor G because this is the meeting of tribes and you have a lot of developers and I'm not a developer and I'm always afraid that people will expect to see things like coding and so on. I had this experience in 2009 in Sydney to play Skyler Earl and he's a very popular guy and I was totally, I'm not expecting different things. So today I would like to talk about this project we are doing in the Met Office, the European project on an important topic in Romania, in Europe and worldwide, the water quality because as you may all know we do have a lot of issues with water quality and the future does not look good. So but first what clean water stands for, so it's the name, this is the acronym of the project but the real name means Integrated System for Protect and Analyze the Status and Trends of Water Threatened by Nitrogen Pollution. The project doesn't focus only on nitrogen but on more other issues on water quality. And this is financed by the European Commission through a framework called LIFE, LIFE Program, a financial instrument supporting environmental and nature conservation projects. And it aims to develop at the basin scale so it's quite like a, not a pilot but it's for just one major basin in Romania. So it would try to create an integrated management system to see when the water quality is threatened and to enable the decision makers to try to simulate what will happen in the future and to take the right decision. So modeling and then assess the, and analyze the results. And of course because the subject of the project has a very, very big geospatial component and we did this, all this component with free and open source software and that's the reason I'm here to present it to Phosphor G. Beside the Met Office we also have some other partners in the project mostly rated with water so the National Institute for Hydrology and Water Management, Romanian waters and so on. Have to speak about our end users because they are national and local authorities and the very job is to make sure the water quality is good. They're in charge and responsible on the good quality state and to be able to assure this water compliance with the national and European laws and there are quite a lot of directives in Europe that deal with water quality is quite an important point. And this decision makers they need instrument to understand the impact of the new industrial or agriculture investment in their water basins. And they do not possess any knowledge whatsoever of how to run a numerical model, a mathematical model to simulate the water quality and they don't know how to use GIS software. We are from different business, they simply don't know how to deal with this kind of technologies. They do know how to use simple tools like Google Earth, Google Maps and this kind of easy mapping frameworks. And of course we are looking for nice tools to be able to create scenarios that involve future water quality improvement. So they need to take decision to improve the water quality but they need a tool to simulate what will be the impact of those measures and also is not only the impact but what is the cost of that measure. So that is why we have to somehow bridge the part of the numerical modeling in situ data collection with some nice easy to use interfaces. Of course geospatial enabled interfaces. And we did this with an online GIS system, distributed architecture that could be run by anyone in the browser. So we could do data visualization, query and spatial analysis, simple one, not very complicated one. They could create new scenarios that involve water quality and send those scenarios to a numerical model. So we basically are able to improve easy to use interface to modify the inputs in the model, send it to the models and then receive the results and have some tools to analyze and see what is the outcome of the measure after the modeling. And of course because you see everything rotates around the money because if you have plenty of money, I don't know, take all the measures necessary to have a good quality state of water, you have to somehow prioritize these. Of course it could be good to use the money in the best way. So you need to have like an analysis of cost effective of the measure you are proposing and to integrate with some popular mapping platforms. So not to talk about numerical modeling, it's not my area of competence, but for this project we are using three types of numerical models. First is for the surface water quality in this model developed by the Pierre-Marie-Quilly University in Paris, France. It's called Sénèque River Strahler. We use mod flow for the ground water quality modeling and there are some in-house development models to model some soil parameters needed for this. And I'm also here in this conference because this project was built on a previous experience, a project called Diminish. And there's a good thing that this project is also an example of migrating from proprietary solutions to free and open source one because the previous system more rudimentary was built entirely on things like ESRI, RKMS and we use Microsoft stack for the databases and for programming language and so on. And then this happened in 2005, 2006 and in 2006 I attended Fast4G in Lausanne and I was amazed by what the free and open source technology can do and I simply for a new project I remove everything we had and this was some example of the previous project and went for the free and open source solutions. So the project starts with some system requirements. Being in this quite complex context we had to take an account of a few initiatives and standards like the Inspire directive which is very, very important in Europe and also we have some satellite data and we somehow have to take into consideration GMEs or the actual Copernicus, the European program for space observation. SASE that is a European shared environment system. We'd like to think that what we are building now is an example of SASE and some directives and the most important is the water framework directive in Europe. And that's some technical stuff. We got these requirements from the end user and taking on the mentioned initiatives and we end up with things like general requirements, data requirements, functionality, security, hardware, software. I'll not bother you with this because it's just what the system should do is like we need interfaces for WMS, for WFS, for all kinds of things, performance things and we got like a big document with what the user needs and then we had to assemble a shared database and of course we got what was already available in different institutions but we had to work a lot almost one and a half year. We did a number of field campaigns and it took I don't know a thousand but many hundreds of field measurements regarding water quality and other issues and it was quite time consuming and we spent quite a lot of time to try to make this GIS database compliant with what the water framework directive is saying and the inspire directive. It should sound easy but it's not, trust me. And this is the content I will not get into details. We also have like data fusion application because we were still receiving data from a number of institutions across the country and we all use different file formats, different platforms and we have to somehow to create a layer that could translate the data we're receiving in some standard formats to be able for our system to use. Of course we make use, besides open software, we do make use of open standards and for this project we are using standards of course issued by OGC ISO and also the standards from Inspire but we are similar ones. So one of the most important are the data portrayal standards like WMS and WMTS. So this is the main view of transporting the data from the GIS server to the clients, the web and desktop clients. And we also like our users to be able to download their data and using it in their own environments so we set up also some download services for the vector based on WFS and for the raster data based on WCS. We have a catalog. We don't have as much data as you may think but at some point it's very good to have a catalog that is also use some standards for the search to be able to be harvest by third party catalogs and we created metadata using some well-known standards. And we also use some light formats like GeoJSON and KML to stream the data. So we use GeoJSON inside our web application. We give the users the opportunity to take the data out as KML because everyone knows how to use Google Earth. So now I'm talking only about the geospatial system architecture with as mentioned a distributed one. We have a database management service, a catalog service, a data delivery and portail service and some or one software client. And the general thing is like it's look like this. So we have the user somewhere could access the system through a web client is how we think most of the project but if they have greater knowledge and they prefer to use their own tools they could use desktop client. So they could connect to our system and make all kind of requests behind. We have the database, the catalog and the services and we stream back the responses to them in different type of formats like maps, flow data, services, charts, animation and so on. And we also have the project partners who are feeding also the system with numerical model simulations and the economic analysis which is quite complicated and I could not really understand it but it has to go there somehow. And if you zoom a little bit on the technology because you are in a the FOSFORG conference. This is the tools you are using. So for the client you are using JavaScript libraries like XGS, GOX, OpenLers and a number of other for float for charts and so on. And we test, we are testing the system with things like QGIS, UDIC but also RGS because some of our end user have RGIS or Google Earth. And then on the service side we have like a normal stack with Apache, Tomcat, Java, PHP and Python for service side processing and then we store all our vector data in a Postgres plus PostGIS database for the roster we are still using the normal plain file like GOTF, compressed GOTF files. Use geo network open source for our catalog and it's a nice product to integrate quite well with other inspire related catalogs in across Europe. And for our GIS server so to be able to deliver data as WMS, WFS and so on. You are using geo server and the geo web cache and as I said the user can get back a number of the results in a number of ways. Now for the final part of my presentation I will present a little bit, little facts about the web client because that's the public face of our system. So it has just a very simple interface. I could say it's very rudimentary. It's not supposed to be very fancy, it has to be effective. So this has two parts, one is for scenario management so place where the user can create scenarios, choose what input data they want to use for the scenario and so on and then we have the normal web mapping application based on open areas in geo X with a main map, some layers, they could switch on and off and just transparency and a number of tools and they could use to query the database there. They could make different information there. They could edit and we spent quite a lot of time for maybe not such useful thing. They could create their own layers or they could edit some of the existing layers. For example to create a scenario it involves to edit the, for example the industrial, the industry in the area. So factories, power plants and so on. So you could delete or add new, this kind of objectives that are very important because they all affect the water quality and they also could change a lot of parameters. So it's not only geometries but attributes because you'll need to know how much, I don't know, the wastewater, how much they have bad water they're sending out. And you also have things here like the, I don't know the name, the station where they are cleaning the water. So again, a lot of parameters and stuff and that's why you're required to do a lot of editing and so on. And you could use a number, we also deploy some WPS services. So for, especially for vector data, the most simple example is to create buffers. So you could select existing elements or you could create your own, I don't know what, with vulnerable areas or whatever, sources of pollution and whatever. And you could do things like buffering and others and then you could create complex selections using also this kind of buffers and the attributes and so on. And of course you could view and analyze the outputs from the models. So this is an example of the groundwater model, you could do a lot of thematic maps with the outputs of the models just to understand what is there. And we have a chart module you could do. So whereas like you could create charts for hundreds of cross sections and what a basin is closing, per basin and so on. You could animate the charts to go in time for one year, for one month, for more than that to go along the river and so on. So most of these guys seems to look a lot of charts. Yeah, understand the impact of the measures they are taking. You could put charts also on the map and so on. You could see it in Google Earth. So for the future work, because the project I didn't mention but it's just past this half time of the project. So we are not done. We have a lot of work to do still. So we have to finalize the integration of the models and the environmental cost because there are some issues there. And we would like to improve the visual look of the system. We want to put our own custom HGIS team. We want to use Bootstrap for all the parts with forms and what the scenarios are created and so on. Because they look pretty pretty. They scale on different devices. You could use it on tablets and so on. Bootstrap is a great technology. We have quite a complicated document for system validation. A number of components has to be validated through using some standard steps. I have to do user training. And we want to open the data. I'm a big fan of open data. And we collect quite a lot of data. And there's not so much data open at this point in Romania at least. And the water quality is an important issue. I think that everyone should be able to access this kind of data and understand how good or bad the water quality is in this area. And this should be important. With this geospatial stack we want to go live no longer than January 2014. And of course have more fun in doing things. So this is the end. Thank you. If you have questions please ask me. We are a little bit behind the time. But that's not my fault. We have enough time for questions. Thank you very much for seeing.
 My name is Vasile Krachunosko, I'm from Romania, from Bucharest, from the National Meteorological  Administration, Remote Sensing and GIS department.  I'm also leading the OSG or Romanian group and I'm always scared in presenting in Phosphor  G because this is the meeting of tribes and you have a lot of developers and I'm not a  developer and I'm always afraid that people will expect to see things like coding and  so on.  I had this experience in 2009 in Sydney to play Skyler Earl and he's a very popular guy  and I was totally, I'm not expecting different things.  So today I would like to talk about this project we are doing in the Met Office, the European  project on an important topic in Romania, in Europe and worldwide, the water quality  because as you may all know we do have a lot of issues with water quality and the future  does not look good.  So but first what clean water stands for, so it's the name, this is the acronym of the  project but the real name means Integrated System for Protect and Analyze the Status and  Trends of Water Threatened by Nitrogen Pollution.  The project doesn't focus only on nitrogen but on more other issues on water quality.  And this is financed by the European Commission through a framework called LIFE, LIFE Program,  a financial instrument supporting environmental and nature conservation projects.  And it aims to develop at the basin scale so it's quite like a, not a pilot but it's  for just one major basin in Romania.  So it would try to create an integrated management system to see when the water quality is threatened  and to enable the decision makers to try to simulate what will happen in the future and  to take the right decision.  So modeling and then assess the, and analyze the results.  And of course because the subject of the project has a very, very big geospatial component  and we did this, all this component with free and open source software and that's the reason  I'm here to present it to Phosphor G.  Beside the Met Office we also have some other partners in the project mostly rated with  water so the National Institute for Hydrology and Water Management, Romanian waters and  so on.  Have to speak about our end users because they are national and local authorities and  the very job is to make sure the water quality is good.  They're in charge and responsible on the good quality state and to be able to assure this  water compliance with the national and European laws and there are quite a lot of directives  in Europe that deal with water quality is quite an important point.  And this decision makers they need instrument to understand the impact of the new industrial  or agriculture investment in their water basins.  And they do not possess any knowledge whatsoever of how to run a numerical model, a mathematical  model to simulate the water quality and they don't know how to use GIS software.  We are from different business, they simply don't know how to deal with this kind of technologies.  They do know how to use simple tools like Google Earth, Google Maps and this kind of  easy mapping frameworks.  And of course we are looking for nice tools to be able to create scenarios that involve  future water quality improvement.  So they need to take decision to improve the water quality but they need a tool to simulate  what will be the impact of those measures and also is not only the impact but what is  the cost of that measure.  So that is why we have to somehow bridge the part of the numerical modeling in situ data  collection with some nice easy to use interfaces.  Of course geospatial enabled interfaces.  And we did this with an online GIS system, distributed architecture that could be run  by anyone in the browser.  So we could do data visualization, query and spatial analysis, simple one, not very complicated  one.  They could create new scenarios that involve water quality and send those scenarios to a  numerical model.  So we basically are able to improve easy to use interface to modify the inputs in the  model, send it to the models and then receive the results and have some tools to analyze  and see what is the outcome of the measure after the modeling.  And of course because you see everything rotates around the money because if you have  plenty of money, I don't know, take all the measures necessary to have a good quality  state of water, you have to somehow prioritize these.  Of course it could be good to use the money in the best way.  So you need to have like an analysis of cost effective of the measure you are proposing  and to integrate with some popular mapping platforms.  So not to talk about numerical modeling, it's not my area of competence, but for this project  we are using three types of numerical models.  First is for the surface water quality in this model developed by the Pierre-Marie-Quilly  University in Paris, France.  It's called Sénèque River Strahler.  We use mod flow for the ground water quality modeling and there are some in-house development  models to model some soil parameters needed for this.  And I'm also here in this conference because this project was built on a previous experience,  a project called Diminish.  And there's a good thing that this project is also an example of migrating from proprietary  solutions to free and open source one because the previous system more rudimentary was built  entirely on things like ESRI, RKMS and we use Microsoft stack for the databases and for  programming language and so on.  And then this happened in 2005, 2006 and in 2006 I attended Fast4G in Lausanne and I was  amazed by what the free and open source technology can do and I simply for a new project I remove  everything we had and this was some example of the previous project and went for the free  and open source solutions.  So the project starts with some system requirements.  Being in this quite complex context we had to take an account of a few initiatives and  standards like the Inspire directive which is very, very important in Europe and also  we have some satellite data and we somehow have to take into consideration GMEs or the  actual Copernicus, the European program for space observation.  SASE that is a European shared environment system.  We'd like to think that what we are building now is an example of SASE and some directives  and the most important is the water framework directive in Europe.  And that's some technical stuff.  We got these requirements from the end user and taking on the mentioned initiatives and  we end up with things like general requirements, data requirements, functionality, security,  hardware, software.  I'll not bother you with this because it's just what the system should do is like we  need interfaces for WMS, for WFS, for all kinds of things, performance things and we  got like a big document with what the user needs and then we had to assemble a shared  database and of course we got what was already available in different institutions but we  had to work a lot almost one and a half year.  We did a number of field campaigns and it took I don't know a thousand but many hundreds  of field measurements regarding water quality and other issues and it was quite time consuming  and we spent quite a lot of time to try to make this GIS database compliant with what  the water framework directive is saying and the inspire directive.  It should sound easy but it's not, trust me.  And this is the content I will not get into details.  We also have like data fusion application because we were still receiving data from a  number of institutions across the country and we all use different file formats, different  platforms and we have to somehow to create a layer that could translate the data we're  receiving in some standard formats to be able for our system to use.  Of course we make use, besides open software, we do make use of open standards and for this  project we are using standards of course issued by OGC ISO and also the standards from Inspire  but we are similar ones.  So one of the most important are the data portrayal standards like WMS and WMTS.  So this is the main view of transporting the data from the GIS server to the clients, the  web and desktop clients.  And we also like our users to be able to download their data and using it in their own environments  so we set up also some download services for the vector based on WFS and for the raster  data based on WCS.  We have a catalog.  We don't have as much data as you may think but at some point it's very good to have a  catalog that is also use some standards for the search to be able to be harvest by third  party catalogs and we created metadata using some well-known standards.  And we also use some light formats like GeoJSON and KML to stream the data.  So we use GeoJSON inside our web application.  We give the users the opportunity to take the data out as KML because everyone knows  how to use Google Earth.  So now I'm talking only about the geospatial system architecture with as mentioned a distributed  one.  We have a database management service, a catalog service, a data delivery and portail service  and some or one software client.  And the general thing is like it's look like this.  So we have the user somewhere could access the system through a web client is how we  think most of the project but if they have greater knowledge and they prefer to use their  own tools they could use desktop client.  So they could connect to our system and make all kind of requests behind.  We have the database, the catalog and the services and we stream back the responses  to them in different type of formats like maps, flow data, services, charts, animation  and so on.  And we also have the project partners who are feeding also the system with numerical  model simulations and the economic analysis which is quite complicated and I could not  really understand it but it has to go there somehow.  And if you zoom a little bit on the technology because you are in a the FOSFORG conference.  This is the tools you are using.  So for the client you are using JavaScript libraries like XGS, GOX, OpenLers and a number  of other for float for charts and so on.  And we test, we are testing the system with things like QGIS, UDIC but also RGS because  some of our end user have RGIS or Google Earth.  And then on the service side we have like a normal stack with Apache, Tomcat, Java,  PHP and Python for service side processing and then we store all our vector data in a  Postgres plus PostGIS database for the roster we are still using the normal plain file like  GOTF, compressed GOTF files.  Use geo network open source for our catalog and it's a nice product to integrate quite  well with other inspire related catalogs in across Europe.  And for our GIS server so to be able to deliver data as WMS, WFS and so on.  You are using geo server and the geo web cache and as I said the user can get back  a number of the results in a number of ways.  Now for the final part of my presentation I will present a little bit, little facts  about the web client because that's the public face of our system.  So it has just a very simple interface.  I could say it's very rudimentary.  It's not supposed to be very fancy, it has to be effective.  So this has two parts, one is for scenario management so place where the user can create  scenarios, choose what input data they want to use for the scenario and so on and then  we have the normal web mapping application based on open areas in geo X with a main map,  some layers, they could switch on and off and just transparency and a number of tools  and they could use to query the database there.  They could make different information there.  They could edit and we spent quite a lot of time for maybe not such useful thing.  They could create their own layers or they could edit some of the existing layers.  For example to create a scenario it involves to edit the, for example the industrial, the  industry in the area.  So factories, power plants and so on.  So you could delete or add new, this kind of objectives that are very important because  they all affect the water quality and they also could change a lot of parameters.  So it's not only geometries but attributes because you'll need to know how much, I don't  know, the wastewater, how much they have bad water they're sending out.  And you also have things here like the, I don't know the name, the station where they  are cleaning the water.  So again, a lot of parameters and stuff and that's why you're required to do a lot of  editing and so on.  And you could use a number, we also deploy some WPS services.  So for, especially for vector data, the most simple example is to create buffers.  So you could select existing elements or you could create your own, I don't know what,  with vulnerable areas or whatever, sources of pollution and whatever.  And you could do things like buffering and others and then you could create complex selections  using also this kind of buffers and the attributes and so on.  And of course you could view and analyze the outputs from the models.  So this is an example of the groundwater model, you could do a lot of thematic maps with the  outputs of the models just to understand what is there.  And we have a chart module you could do.  So whereas like you could create charts for hundreds of cross sections and what a basin  is closing, per basin and so on.  You could animate the charts to go in time for one year, for one month, for more than  that to go along the river and so on.  So most of these guys seems to look a lot of charts.  Yeah, understand the impact of the measures they are taking.  You could put charts also on the map and so on.  You could see it in Google Earth.  So for the future work, because the project I didn't mention but it's just past this half  time of the project.  So we are not done.  We have a lot of work to do still.  So we have to finalize the integration of the models and the environmental cost because  there are some issues there.  And we would like to improve the visual look of the system.  We want to put our own custom HGIS team.  We want to use Bootstrap for all the parts with forms and what the scenarios are created  and so on.  Because they look pretty pretty.  They scale on different devices.  You could use it on tablets and so on.  Bootstrap is a great technology.  We have quite a complicated document for system validation.  A number of components has to be validated through using some standard steps.  I have to do user training.  And we want to open the data.  I'm a big fan of open data.  And we collect quite a lot of data.  And there's not so much data open at this point in Romania at least.  And the water quality is an important issue.  I think that everyone should be able to access this kind of data and understand how good or  bad the water quality is in this area.  And this should be important.  With this geospatial stack we want to go live no longer than January 2014.  And of course have more fun in doing things.  So this is the end.  Thank you.  If you have questions please ask me.  We are a little bit behind the time.  But that's not my fault.  We have enough time for questions.  Thank you very much for seeing.
Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__0_1511.png Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__1512_1679.png Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__1680_1727.png Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__1728_1931.png Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__1932_11339.png Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__11340_11483.png Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__11484_11663.png Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__11664_11807.png Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__11808_16175.png Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__16176_17819.png Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__17820_22223.png Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__22224_22283.png Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__22284_22415.png Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__22416_22559.png Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__22560_26267.png Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__26268_27935.png Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__27936_28055.png Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__28056_28367.png Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__28368_30287.png Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__30288_30347.png Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__30348_30407.png Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__30408_30455.png Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__30456_30527.png Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__30528_30587.png Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__30588_30659.png Water_Quality_Forecast_And_Information_System_Built_With_FLOSS_Software__720p__30660_30686.png