GEE toolbox- efficient computing using Google Earth Engine and R
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Parente, Leandro Leal
This talk is about Google Earth Engine-GEE, a cloud-computing platform capable of processing petabytes of Earth science data and presenting the result of planetary-scale analysis on-the-fly. You will learn about the main concepts, its architecture and the best practices to incorporate GEE in your workflow. For this, I will present a hands-on, using R (rgee), to generate a land-cover classification for a Landsat scene in the Amazon forest (226/67), which will considerer all the landsat images obtained for 2019, a Random Forest model and a hyperparameter optimization implemented in MLR3.
 Hi everyone, good afternoon. So first, I want to say that I'm sorry for not being physically in the summer school, but I'm following the event and my internet and I believe you are having good times in bargaining. So let's start. I will start my presentation. So my name is Leandro Parent. I'm a postdoc research at OpenGL Hub Foundation. And today I will talk about Google Earth Engine and how we can efficiently use it in R. And my presentation, my talk is organized in three parts. So the first part is about the platform, the Google Earth Engine, an introduction about this outstanding platform and how we can integrate R with it. And in the second part, I will do a code demonstration in the R studio. So we will run a land cover classification for a land sat-scene located in the Amazon forest. And in the last part, I will summarize some concluding remarks. So first, what is Google Earth Engine? For me, Google Earth Engine is a free cloud computing platform capable to analyze on-the-fly science data in a planetary scale. So it's important to say that free doesn't mean free software here. Google Earth Engine is a business solution developed by Google. But anyone can sign up and start using the platform, run your own analysis and upload your own data in the Google infrastructure and take advantage to that in your own workflow. And other important aspect is this platform promotes on-the-fly planetary analysis. So it's a great functionality because we can easily access all the data from, for example, land sat or sentinel in any part of the world. And we can visualize the results in some seconds. So it's a great functionality. So basically anyone that works with Earth observation data knows that we need to download this data from several Earth observations, from several providers to our own server, to our own Earth station or our own server hosted in a cloud platform. But we need to download the data to start, run our customized algorithms and analysis. With Google Earth Engine, we need to think in an opposite way. So basically we upload the code to the Google Earth Engine and we run some specific analysis in that infrastructure, that cloud infrastructure, to process several Earth observation data organized in one place. So we can scale up any type of analysis, for example, to the whole world using the same code. So it's an important aspect in this type of paradigm, namely moving code. So what are the public available data sets in Google Earth Engine? So we have all the land sat archive since 85. We have sentinel 1, 2, and 3. We have SRTM, ALOS, Open Landmap, Soil Data, and several climate data sets. And we can consult, we can access and search in this catalog through this link. So you can easily discover if some specific data set is available in there. And if not, you can upload it by your own or you can ask to Google team to upload and to insert it in the platform. And about the architecture. So basically Google Earth Engine is a cloud computing solution. So we need to interact with the platform by client libraries. So there are two official client libraries developing JavaScript and Python. And using this library, we construct this script. We construct like a representation, an object representation of our own analysis. And we upload, we submit it to the cloud platform to process and to run in the Google infrastructure. So basically we receive the output of this analysis by two ways, by on the fly. So we can analyze and visualize the result in a web master. Or we can export it to a batch computation directly to Google Drive or to Google Cloud Storage. So here we can see an example of this on the fly analysis. So this is the annual median for NDVI considering all Landsat 8 observations for 8 for 99. For 20 in 19, 200 and 2, 2, 2, 1, 19, sorry. And this data is a surface reflectance data. So basically if you click in this code, you will access the JavaScript code in the code editor. And you can access the code editor in the code editor. And you can visualize the output of this analysis on the fly way directly in your browser. And basically what is happening here. So in this analysis, I removed all the clouds and cloud shadow pixels. So each pixel that has a cloud shadow, it's a cloud shadow. So each pixel that has some type of contamination, I removed it from all the images available in Landsat in 2019. And I got this time series with a lot of gaps. And after that, I apply some type of temporal reduction by median. So basically in this specific example, we have 0.4 as output result for one specific pixel. But what I implemented here is a specific way to reduce all the images obitained for a specific year to one image. So I reduce one image collection to only one output. So we can apply several types of reduction. So this is a very common operation in the platform. And you can access it directly by this link if you have the access to Google Earth Engine. And you can check it for any part of the planet as I calculated it for the whole work. And considering that a good way to work with Earth Engine, we are doing it in OpenGL Hub. So a good way is use the Google Earth Engine in part of your workflow. So here, for example, I can upload this code to generate a median annual NDVI image for the specific year. And export only this output. So I process all the Landsat archives for a specific year and generate only one output to Google Drive or to Google Cloud Storage and download this result to run other types of analysis in our own servers. So a good example of that is it. So here we can see all the NDVI data for the whole work obitained by AVHRR for the year of 1991. So this is purely Earth Engine. If you click in this link, you will have access to the code that generates this GIF and this animation. And what we did at OpenGL Hub, we downloaded all the monthly NDVI data for the whole work. And we downloaded it from AVHRR data and from Moldes data. And we run locally iSmoot and I get a few approaches using this green Brawl package. So this is a good example of how we can actually run and incorporate Google Earth Engine in one workflow because they organize it, they run the monthly analysis for this whole period was implemented in Google Earth Engine and we only downloaded all the monthly NDVI data for several years. And about R, we need to know about it because Google Earth Engine only provides JavaScript and Python as official client libraries. So to integrate the platform with R, we could use this package in the code demonstration. We will use this package, the RGAAA. It's an official Earth Engine API, but a great piece of software. And what this package implements is an integration between the Python API, the official Python API for Google Earth Engine through Reticulate. And with this package, we can access on-the-fly computation in the RStudio and we can export the output of our own analysis to the Google Drive and to Google Cloud Storage and integrate it with the R spatial ecosystem. So what you need to know about this type of integration? So first, you need to sign up for Google Earth Engine. It's free. You just need to click in this link and fill the form. After that, you need to install the RGAAA package by a virtual envy or a Docker image. There are two options to do that. After you need to set up and authenticate in the Google Earth Engine, in Google Drive and Google Cloud Storage, Google Cloud Storage and Google Drive is official. And I suggest to you read the best practice because in this link, you will find a good material to understand the difference between the Google Earth Engine and the Earth Environmental. So there are some differences that you need to know before start coding. And a good starting point is these R examples. So here you will find several examples that were based in an official user guide and were rewritten in the R. And one important tip is that the official Google Earth Engine is only available in JavaScript. So to know the name of the methods, the name of the parameters, you need to access the code editor and check in the JavaScript. But you will have access to all these methods and all the parameters in our M-Python. So let's start our code demo. So first, we need to know what we will do. We will run a LAN-Cover classification for one LAN sat-scene in the Amazon Forest. We will work with predefined samples that I collected for that area. We will use these samples as an uploaded asset in the Earth Engine. After we will run a temporal reduction overlay samples and we will export the result of the overlay. So the pixel value for each sample that was calculated over LAN sat data through Google Drive. And after that, we will run a Hondon Forest tuning approach using MLR tree. I won't talk about too much about MLR tree because I don't have time enough, but it's a great piece of software too and you need to check it. I suggest you to check in the website, in the official website. And after that, we will run a classification for the whole area considering all the LAN sat-scenes and export the classification result. So you will find the code here. So if you click in this link, you will find the presentation, the samples, and the code that I will run in R. So I put here other extra file with some type of analysis and some suggestions to configure the environment in the Earth Studio. And if you access, I will access only quickly the code editor here to show that we will use this asset. So I uploaded the CSV file to Google Earth Engine so you can upload the GLTIF shape file, CSV file. I uploaded my CSV file to Google Earth Engine and anyone can read it because I share this asset with a public. So this asset is public. So we will access the samples that are stored inside of the Google Earth Engine in R. So first, I will start with these helper functions. I will talk about them in the next minutes. Can you please increase a bit the font size in RStudio and the tools global options. And the appearance. Yeah, and then you edit the font size. Yeah, put like 18 or something like this. They want also the team to change into white. It's easier to read. Yeah, it's difficult to read on the screen. Yeah, all right. I will change here. That's the, yeah, something like this. Yeah, Chrome is fine. Okay, great. Much better. Thank you. All right. So I will start with these helper functions that I will talk about in the next minutes. But basically, I wrote these functions to help in this code demonstration. And I will load the RGAA package. You need to configure it in your own machine. There are information. You will find this information in the official website. But in the extra, you will find the dependencies that I installed and what are the functions that you need to run to set up, for example, the authentication. You just need to run it one time. And to start the code demo, the code demonstration. First, we need to initialize the connection with Google Earth Engine. As I said, Google Earth Engine is a cloud computing platform. So we need to establish a first connection with the platform to start and use it in R. And I will access my samples that I uploaded in the Google Earth Engine through this function. So basically, future collection is a representation of vector data in the Google Earth Engine. And this is the ID, the unique location of some data set that is provided in the Google Earth Engine. And we need to, the first thing we will run is what is the number of the samples that we have here. So it's an important part because if you run only this code here, sample size, you will get only this Earth Engine object number. So basically here, you don't have a result of the output, the analysis method because in this part of the code, we only have a description of some type of analysis. So basically, we need to send this description to the platform to run in the Google Cloud infrastructure and after that download or get the result. You only will have the result if you call getInfo. So basically, if you want to access any type of result in considering this method, considering these functions, you need to call the getInfo to get the real result of analysis forcing the API to access the Google Earth Engine and get the result from the cloud. So this is one type. We will have the access of the data to the viewer. That is the other type. So we will access the samples and we will see it in a web map. I will show in the next minutes. So what are the sample attributes? So basically here, we will see that we have an ID and a proper name it LandCover containing the values of LandCover for the sample. And in this toy sample, we have a forest. We have samples for forest, basher land, crops, others, and water. So basically, I will create a list with one color for each of these classes. And I will use this function that I implemented to plot all the samples separate then by colors. So basically here, we are seeing the all the samples organized by layers, by class, and the number of samples that we have in the each class. And we are seeing, so this is stored in Google Earth Engine and now we are seeing the results in our studio. So it's some type of the on-the-fly visualization. Basically, this function is responsible to split the samples and apply the color in the using the parameters of the Google Earth Engine. What we will do now is calculate and visualize the sample extent. So you can see that I'm reducing all the geometries of the samples and I'm calculating the bounds, so the extent of that sample. And I will put this result, this output in a layer and I will put the layer in a list of layers. And if I send to my function this layers object, it will plot the samples and the what we have in the list. So basically, we are seeing here the extent of our samples. So this is a bounding box and it's important because we will use it to refine, to specify what is the spatial extent for our analysis. So basically, to run this analysis, we will reduce all the images of Landsat data, considering surface reflectance product, obitained for the whole year of 2019. I will filter by this tree, by this old spectral bands and I will pass this extent. And this is an important parameter because this is a mathematical representation of our valid pixels. So Earth Engineery will use it to remove the cloud and cloud shadow pixels using the Landsat quality band assurance. So to understand these values, you should access this official documentation by US GIS to understand what are the values that are maintained here. And I will run a reducer. So we can access, we have several reducers like standard deviation, median, but in this case, I'm running the percentile 25, percentiles 50, and the percentile 25. So basically, I will, so I need to create this. So basically here, we are only, we only have the descriptor. We don't have all the Landsat data here in my notebook, but we can visualize it in the viewer when we put it on the map. So for example, if I call this function band names and I get info, so I force it now, the API to access the real band names in the Google Earth Engineery and retrieve to me in the object representation here in car. So by default, Google Earth Engineery don't provide any type of enhance, so automatic visualization for Landsat data. And I create this function to basically calculate the minimum and the maximum values for each of specified bands. So it's only a function to define the visualization parameters that will be used in the map. So now we are seeing, we will see the annual median for all the spectral bands in the, our interesting area. So we can check, for example, if the forest samples are in the right place and things like that. So we have all these. So you calculated these satellite bands, you calculated them in Google Earth Engine, right? And then you now load it, you display it in R. Is there any like a temporary object that you download from Google Earth Engine to your machine? I mean, or is it just really loads it in the memory? How does it work? No, it's, it's, it's worked really in on the fly here. I only have access to a tile server that is provided by Google Earth Engineery. So all the analysis were processed in Google Earth Engineery in here. I only have a visualization of depth, for example, in PNG. And depth is available in the web map. Okay, so you don't have the actual data? You get the data locally, you just got the PNGs from the, okay. Yes, okay. But it's important thing because, for example, I can check it and I can see that that image has a lot of clouds because my, for example, my valid expression, my mathematical expression actually didn't remove the cloud. So it's good to have this type of representation before to download the data. Okay, continue, please. So what we will do now, so I, I calculated it for all these Landsat bands. So it's 21 bands. And now, now I will export the data. So what this function implements, these, this function extracts the Landsat percentile values for each sample, export it as a CSV file to Google Drive, download it from Google Drive and load it in R. So basically, here, I sent to the function the samples and my image. This image is not loaded in the, in the, it's not in my memory. It's in the Google Earth Engine. So I process all the things in the Google Earth Engine and I export it on the output as a CSV file. And now I have the real output here in my memory. So just like a CSV file. So I have all the values for the samples. And basically, I have, for each spectral band, I have one column. And I can access it right now. If you had, if you had like a million, million points, and if you had to overlay like thousands of clusters, that could be computational, right? And yes, yes. And Google block you maybe if you send to too big number, I mean, how does it work? To, to work with this number of points, my suggestion is don't use this function, only export the output to Google Drive. So basically, if you look in this function, this is an exportation task. So basically, you can send like the name of the samples, the name of feature collection stored in Google Earth Engine, you can send it to the platform and export is it to Google Drive. So basically, you just need to wait the exportation. Sometimes we'll take more than one hour, maybe 10 hours. But you, you have, you will have the result in Google Drive. And after that, you can download it. So basically, now we will filter this, this data frame to use only the bands that we are interested. So we will have the land cover, the predicted variable and the, and, and the all the spectral values that was extracted. And I will convert it to factor to running MLR tree. And basically, I will run my hyper parameter tuning using MLR tree. There are several options to implement it. I won't talk about too much MLR tree and you can find more information in this link. But what I will do now is I will run the optimization and explain the code. So basically, now I have all the samples here in my machine loaded in or and I will create a classification tasks. It is a basic object in MLR tree. So I'm sending to the MLR tree my classification matrix and what is the variable, what that I'm interesting. So my target variable here, I'm using random forest. So it's important to use random forest because after this hyper parameter optimization, I will classify the entire scene using Google Earth Engine. So I need to have the same algorithms and the same parameters implemented in Google Earth Engine because I don't have the Landsat data in my, my local notebook. So I, I'm only running a hyper parameter optimization here. And your model is fitted in R and a Google Earth Engine, it doesn't have an R engine. So how do you export the model to Google Earth Engine? It's not possible. I need to retrain it in Google Earth Engine. Basically, what, what I will have here is the best hyper parameters for classify random forest for this specific area. Yes. And the Google Earth Engine doesn't have this functionality of MLR tree to do the fine tuning, right? So you do the fine, only fine tuning locally, then you get this fine tuned like a one or two parameters and then you use that in the Google Earth Engine. That's what you're saying. Okay. Exactly. And so in Google Earth Engine, I can only choose three hyper parameters, minimal leaf population, number of trees and variable per split, if I consider random forest. So basically what I'm doing here is finding the best hyper parameters in MLR tree to apply it in Google Earth Engine, retrain my algorithm and classify the whole, the whole area. I, I, I, I, I define, define this design. So here we will have all the available parameters that will be tested in MLR tree. So basically, we can, I think it's more than 2,200 parameters. And I will show it when the optimization has finished. And I created this tuning instance to define how it will work. So basically what happens here is the, the tuning will consider five fold cross validation to maximize the overall occurrence and will test all the values that I established here in this design. So if I look it, I have all the possibilities of the parameters, of hyper parameters here. And this function will test all and evaluate it by overall occurrence. And basically after that, we need to create a tuner object and start the optimization approach. And the output is presented here. So we can see that we have 86% of overall accuracy considering all the class. And we have these three values for the hyper parameters. And now we can try one, try one, it's a bit of a problem. It's, it's too small. Yes, but, but basically it's compensated by, for example, the number of trees. Because what happened here, he tested all the combinations and the best overall accuracy was obtained with that. Of course, this, this could be, for example, this could, could happen because we have fewer samples. So here we have almost 40, 400 samples. Probably if you have more samples, it will have different results, but we can visually compare it with, for example, the full parameter in Google Earth Engine. And we can see if it really produce a better result. Okay, continue, please. So basically, and other possibility here is we can see, this is an important feature functionality that doesn't exist in Google Earth Engine. We can see that what is the best and Landsat metrics that are contributing to the classification approach. So the best metric is the SWIR 2 percentile 75. So we can see this Hank for all the variables. And here we have the three optimized values. So basically now we are ready to run the classification in Google Earth Engine. And what I will do is I will overlay again the points because it needs to run in Google Earth Engine. So if you check this, this is a function that overlay inside Google Earth Engine without export it. If you check it, it's just a descriptor. The whole thing will be processed only when I show something in the map or only when I call a get info function. So in this case, it will only run on the fly when we see the, we see the result in the, in the air studio in the map. So basically, I will create two random forest models. So one using 103 and the default parameters and other using the MLR tree optimized hyper parameters. And I will train these two models. Again, this is not, this is only a descriptor. The whole thing will be processed when I put these results in the map. So I'm here, I'm creating a training order to, to classify the, to create a model. And after that, I will classify the same images using two different models. And I will put all the things, all these layers inside Google, inside the, the map to analyze and to see the result. So I like this visualization here. It's better. So now all, all the processing here are, is, is running on the fly. So basically we have two classifications that were trained and predicted in the, this Landsat dataset. And we can compare, visually compare the results. So this is the MLR tree optimized hyper parameters. And this is the default classification with the full hyper parameters of Google EarthFansion. And basically, and we can compare it to the real data in using the Landsat and medium image. So basically we can zoom in in some part and we can check if the result is what is the difference between the results. So basically we can see more in some places we can see that they're more consistent. But again, this is only a toy example. You need to create some different analysis, probably some parts we lost some water, but we have a good convergence, a good spatial consistent, considering, for example, even high resolution data like this, for example. So we are seeing the result. And now we can export it. So we have a classification object and we can export it to Google Drive. And we can start exportation test. It's possible to export the Landsat medium image. So we can export it to Google Drive too. And you specify, for example, some last bands, then, for example, all the bands that were calculated. And if we check in the code editor, basically, I will reload here, all the exportation tasks that were started in the R we can see here. So it's basically it's running on the Google EarthFansion. And I will get the result in the Google Drive. So I download the data so you can access the final results in this folder too. And it's possible to visualize it in the GIS. So yes, this is the this is my code demonstration. And to summarize, you can try yourself these two possibilities of analysis. So, for example, you can expand the feature space using more percentile order reducers. And you can use Sentinel to data in the same area with the same samples to get, for example, different map using a better data. So to summarize the things, all the points, the use of Google EarthFansion by scientific community is increasing. So basically, in the last years, we have several publications in peer review journals that are using Google EarthFansion in different study areas. And we have from some special issues in peer review journals too. And in my opinion, Google EarthFansion is an outstanding platform that allows users to test large scale, geospatial analysis quickly and on the fly. And for me, the best way to analyze to run it to use Google EarthFansion in your workflow is exporting a final result or intermediate result as we did here. And we you can export route images, for example, like for all the Landsat images for one year, but it's not recommended and will take too much time. And you can use multiple Google accounts to operate together and write in a shared bucket hosted in Google Cloud Storage. So it's an example of that. It's, for example, a land cover classification for the whole country or the whole continent. You can have several accounts sending a classification request to write in the same bucket. And it's possible to use Google EarthFansion to serve images as web GIS application. And you can try to join to Gel for Good Summit. It's like an annual summit for Google EarthFansion users. And they launch, for example, the new functionality in this event. So it's a good event to understand better the platform and what will be the news. And sorry to be a little harsh, but my goal was to present and to end an analysis and an example to integrate Google EarthFansion with R. All the code is available. I can respond some emails to if you have other questions later. And now we have time for the questions. Thank you. Okay, thank you. Okay, we do have time for questions. So I think that was a very good demo. Giuseppe, we'll start. Please tell the question. I'll repeat. Yeah. So in Google EarthFansion, can you use random forests for regression problems? Yes. You have several models available in Google EarthFansion. And you can specify if you run it for regression or for classification. So this, you can see here, all the, you can see here, all the in the classifier, you can see all the available models, but you can specify if it will be a regression, I think it's in the output. So you could specify if it is a classification problem, a regression problem, or a probability. For example, for binary classifications, you have the possibility to export the probability for each class. So even though function is called classifying classifier, it actually can be used for regression problems too. So it's a bit hidden. Yeah, it's misleading. But yes, it's possible. Yes, it's a pity that, you know, you, because you do an MLR, you already do the work, right? You do this optimization, and that you cannot then use exactly that model you got from MLR that you cannot deploy it in Google EarthFansion. And, and that's because, yeah, I don't know this, is there any intention to make Google EarthFansion, you know, compatible to more, like more open source solutions or? No, that's, that's really a good question, because actually, we can't extend Google EarthFansion. So for example, we can implement some native function here, we only can use it. So did so the decisions are totally dependent from Google team. So basically, I, I use Google EarthFansion for a few years, but the whole concept here, it's not like a deployed model, but run the whole team. Again, so for example, even if I had the, the, the, my model, train it and I save it, I can't put it in, in Google EarthFansion to run, for example, for other area, I need to retrain the model and export only the output. So all the team need to run on end to end again. But I know that for deep learning approach, they have some type of cloud solution named ML cloud, I think, that you can host, for example, a deep learning, a neural network, train it in TensorFlow in there. And after that, you can call it from Google EarthFansion. But I think it's just for a neural network, for other classical machine learning approach, I don't see anything. Okay. And there was a question, somebody asking Q&A, somebody asked about the bottom neck. So like, let's say if you, if you have more points or something, then slowly or Google Drive, I think it's the, on the free account, it's something like 30 gigabytes or something. So then your Google Drive runs out of space, and then you, you will, you will experience the storage problems, right? Yes. Actually, if you really want to use Google EarthFansion for a huge analysis and generate big images, you need to, the best option is use Google Cloud Storage, that you will have more option, but you can expand your Google Drive account too. But for more production, to produce a big Earth observation result, you need to run some type of the cloud solution. And something I didn't mention. So here you have a personal asset. So this is your asset code. So you have for free, this have this amount of gigabytes available. So for example, any analysis that I want to only access in Google EarthFansion, I can rewrite it in Google EarthFansion. So basically, for example, if I don't need this, the output in my local machine, I can rewrite the output in the Google EarthFansion to access it in a more quickly way, as like, for example, our regular Landsat image. But in general, you want to, to recover the data and export it from, from Google EarthFansion to work in other platforms. But it's a, it's a good option, for example, to split parts of your process. You can process, for example, some NDVI calculation and generate the median as assets. And after that, you can train using the output of the first analysis. And so like Microsoft has the cloud services for like R to compute in R to deploy R. And if, does Google has something like that that you deploy like R and then, and maybe then you could connect Google Earth Engine and, you know, some R installation, do you think something like this could work? Yeah, I, I didn't see anything like this, like this. But probably, just a second, as I think you can help here. One option there is that if you run a server in Google Cloud, and if you run a P-L on that, you can get these out with your Rintian driver and you can directly read files from the EarthFansion archive. So you can do your own processing in your own Rented Service, reading from the Earth Engine archive. That's all processing is an outside Earth Engine. You don't use Earth Engine processes if you use the data that's there. GDAL has an Earth Engine driver. Did you hear that? Did you get that, Landro? So only the last part that GDAL has an Earth Engine driver. It's right. Yes. So the idea is that you use the Google Earth Engine functionality in the cloud installation. Ah, okay. Okay, okay, okay. So from Google Cloud, you can access the raw satellite images. And so basically, you still do computing inside Google infrastructure, but you don't use the Google Earth Engine. Yes. But you use the data from the Google Earth Engine. So that's, that could be a solution, like the backend solution to compute your own way with the Google Earth Engine data. Okay. No, okay, good. That's good to know. Okay. Are there any more questions? Giuseppe? Did you hear that? That you can mount the Google Drive? No, no, I couldn't hear. Sorry. Okay, I will repeat. So Giuseppe said that you can mount the Google Drive inside your server. And then if you run out of space, so you can move the data, so you never run out of space. You continuously move the data, so you always keep it within the limit of 350 gig or something. Yes, yes. That could be a trick. Yeah. Are you cool? Yeah. With the R clone. Yes, you can export the result, download it and erase. So you can use only, you can use Google Drive only as temporary storage to get the outputs from Google Earth Engine. So it could work. And if this package, it's really easy to implement that because RGAA provides access to Google Drive using an API, the official API. So it could be a good work to, a good way to implement it. Okay. More questions? Yes, in the back, please speak loudly. Okay. So there's a ee ee underscore s underscore raster function to export the image and get it locally, right? And that's in the RGAA package. Yeah. Okay. Okay. So there is already some wrapper to get the images, but then you depend on your bandwidth and download the whole world at 30 meters. Just a little chunk. Okay. More questions? So you mentioned also this visualization. So once you make the analysis, you know, you've been using the leaflet, but you know, it's useful to create kind of like a web mapping up. So you put some HTML or something somewhere and then people can access it without needing to run the code and or, you know, use R. So tell us something about that. How complicated is that? How much coding do you still have to do? And is that something that you can do then from R or or you have to use some Google or Tange in? Well, actually, to integrate a Google or Fange with any type of the type of web GIS solution, you need to create like access token to get the result for any specific analysis. So for example, for the result of annual and NDPI or for specific Landsat data, but you need to refresh this token, for example, every day. So I did that. So you need to create like a publication link through the Python API and have access to the to our URL and to a token that you provide to you like tile servers, tile servers to access that analysis. And you can put it in any type of application like leaflet application or open layers application. But you need to handle this token. So a good way to do that is republish all the day and refresh that token. So I think I put here in the there is an application named TD lights. I didn't remember. But these applications show how integrate Google or Fange any with any type of web applications. So yes, this one. So mostly you need to publish the layer using Python. And you can access the result trendy light. So this one. So you can access the output of this analysis analysis to a tile service API that's that is provided by her fanging. But it's not easy, but it's possible. Okay. Yeah, we have the on Thursday, we have a team talking about the map view. So you're going like few lines of code, you create a web map, you know, you customize it and you you set up things that will be useful to make also for Google or attention to like quickly create this web mapping apps. But you said you need a token and it's a bit more complicated. Yeah, but but yeah, thinking better now, I think our other other possibilities like for example, here, I'm using the RG a package. So I only create this map representation that's probably is using map review. So you can integrate it with the ecosystem, because all the token creation and all the steps that I described probably is implemented in this function at layer that is that is implemented by our GBA. So probably could be option using this output of this function here. So this is a map view instance to extend the visualizations in the art.
 Hi everyone, good afternoon.  So first, I want to say that I'm sorry for not being physically in the summer school,  but I'm following the event and my internet and I believe you are having good times in bargaining.  So let's start. I will start my presentation.  So my name is Leandro Parent. I'm a postdoc research at OpenGL Hub Foundation.  And today I will talk about Google Earth Engine and how we can efficiently use it in R.  And my presentation, my talk is organized in three parts.  So the first part is about the platform, the Google Earth Engine, an introduction  about this outstanding platform and how we can integrate R with it.  And in the second part, I will do a code demonstration in the R studio.  So we will run a land cover classification for a land sat-scene located in the Amazon forest.  And in the last part, I will summarize some concluding remarks.  So first, what is Google Earth Engine?  For me, Google Earth Engine is a free cloud computing platform capable to analyze on-the-fly  science data in a planetary scale.  So it's important to say that free doesn't mean free software here.  Google Earth Engine is a business solution developed by Google.  But anyone can sign up and start using the platform, run your own analysis and upload  your own data in the Google infrastructure and take advantage to that in your own workflow.  And other important aspect is this platform promotes on-the-fly planetary analysis.  So it's a great functionality because we can easily access all the data from, for example,  land sat or sentinel in any part of the world.  And we can visualize the results in some seconds.  So it's a great functionality.  So basically anyone that works with Earth observation data knows that we need to download  this data from several Earth observations, from several providers to our own server,  to our own Earth station or our own server hosted in a cloud platform.  But we need to download the data to start, run our customized algorithms and analysis.  With Google Earth Engine, we need to think in an opposite way.  So basically we upload the code to the Google Earth Engine and we run some specific analysis  in that infrastructure, that cloud infrastructure, to process several Earth observation data  organized in one place.  So we can scale up any type of analysis, for example, to the whole world using the same code.  So it's an important aspect in this type of paradigm, namely moving code.  So what are the public available data sets in Google Earth Engine?  So we have all the land sat archive since 85.  We have sentinel 1, 2, and 3.  We have SRTM, ALOS, Open Landmap, Soil Data, and several climate data sets.  And we can consult, we can access and search in this catalog through this link.  So you can easily discover if some specific data set is available in there.  And if not, you can upload it by your own or you can ask to Google team to upload and to insert it in the platform.  And about the architecture.  So basically Google Earth Engine is a cloud computing solution.  So we need to interact with the platform by client libraries.  So there are two official client libraries developing JavaScript and Python.  And using this library, we construct this script.  We construct like a representation, an object representation of our own analysis.  And we upload, we submit it to the cloud platform to process and to run in the Google infrastructure.  So basically we receive the output of this analysis by two ways, by on the fly.  So we can analyze and visualize the result in a web master.  Or we can export it to a batch computation directly to Google Drive or to Google Cloud Storage.  So here we can see an example of this on the fly analysis.  So this is the annual median for NDVI considering all Landsat 8 observations for 8 for 99.  For 20 in 19, 200 and 2, 2, 2, 1, 19, sorry.  And this data is a surface reflectance data.  So basically if you click in this code, you will access the JavaScript code in the code editor.  And you can access the code editor in the code editor.  And you can visualize the output of this analysis on the fly way directly in your browser.  And basically what is happening here.  So in this analysis, I removed all the clouds and cloud shadow pixels.  So each pixel that has a cloud shadow, it's a cloud shadow.  So each pixel that has some type of contamination, I removed it from all the images available in Landsat in 2019.  And I got this time series with a lot of gaps.  And after that, I apply some type of temporal reduction by median.  So basically in this specific example, we have 0.4 as output result for one specific pixel.  But what I implemented here is a specific way to reduce all the images obitained for a specific year  to one image.  So I reduce one image collection to only one output.  So we can apply several types of reduction. So this is a very common operation in the platform.  And you can access it directly by this link if you have the access to Google Earth Engine.  And you can check it for any part of the planet as I calculated it for the whole work.  And considering that a good way to work with Earth Engine, we are doing it in OpenGL Hub.  So a good way is use the Google Earth Engine in part of your workflow.  So here, for example, I can upload this code to generate a median annual NDVI image for the specific year.  And export only this output. So I process all the Landsat archives for a specific year  and generate only one output to Google Drive or to Google Cloud Storage and download this result  to run other types of analysis in our own servers.  So a good example of that is it. So here we can see all the NDVI data for the whole work  obitained by AVHRR for the year of 1991. So this is purely Earth Engine.  If you click in this link, you will have access to the code that generates this GIF and this animation.  And what we did at OpenGL Hub, we downloaded all the monthly NDVI data for the whole work.  And we downloaded it from AVHRR data and from Moldes data.  And we run locally iSmoot and I get a few approaches using this green Brawl package.  So this is a good example of how we can actually run and incorporate Google Earth Engine in one workflow  because they organize it, they run the monthly analysis for this whole period was implemented  in Google Earth Engine and we only downloaded all the monthly NDVI data for several years.  And about R, we need to know about it because Google Earth Engine only provides JavaScript  and Python as official client libraries. So to integrate the platform with R, we could use this  package in the code demonstration. We will use this package, the RGAAA. It's an official Earth  Engine API, but a great piece of software. And what this package implements is an integration  between the Python API, the official Python API for Google Earth Engine through Reticulate.  And with this package, we can access on-the-fly computation in the RStudio and we can export  the output of our own analysis to the Google Drive and to Google Cloud Storage and integrate it with  the R spatial ecosystem. So what you need to know about this type of integration? So first,  you need to sign up for Google Earth Engine. It's free. You just need to click in this link and  fill the form. After that, you need to install the RGAAA package by a virtual envy or a Docker image.  There are two options to do that. After you need to set up and authenticate in the Google Earth  Engine, in Google Drive and Google Cloud Storage, Google Cloud Storage and Google Drive is official.  And I suggest to you read the best practice because in this link, you will find a good  material to understand the difference between the Google Earth Engine and the Earth Environmental.  So there are some differences that you need to know before start coding. And a good starting  point is these R examples. So here you will find several examples that were based in an  official user guide and were rewritten in the R. And one important tip is that the official  Google Earth Engine is only available in JavaScript. So to know the name of the methods,  the name of the parameters, you need to access the code editor and check in the JavaScript.  But you will have access to all these methods and all the parameters in our M-Python.  So let's start our code demo. So first, we need to know what we will do.  We will run a LAN-Cover classification for one LAN sat-scene in the Amazon Forest.  We will work with predefined samples that I collected for that area. We will use these  samples as an uploaded asset in the Earth Engine. After we will run a temporal reduction  overlay samples and we will export the result of the overlay. So the pixel value for each sample  that was calculated over LAN sat data through Google Drive. And after that, we will run a  Hondon Forest tuning approach using MLR tree. I won't talk about too much about MLR tree because  I don't have time enough, but it's a great piece of software too and you need to check it.  I suggest you to check in the website, in the official website. And after that,  we will run a classification for the whole area considering all the LAN sat-scenes and export  the classification result. So you will find the code here. So if you click in this link,  you will find the presentation, the samples, and the code that I will run in R. So I put here other  extra file with some type of analysis and some suggestions to configure the environment  in the Earth Studio. And if you access, I will access only quickly the code editor here to show  that we will use this asset. So I uploaded the CSV file to Google Earth Engine so you can upload  the GLTIF shape file, CSV file. I uploaded my CSV file to Google Earth Engine and anyone can read it  because I share this asset with a public. So this asset is public. So we will access the samples  that are stored inside of the Google Earth Engine in R. So first, I will start with these helper  functions. I will talk about them in the next minutes. Can you please increase a bit the font size  in RStudio and the tools global options. And the appearance.  Yeah, and then you edit the font size. Yeah, put like 18 or something like this.  They want also the team to change into white. It's easier to read. Yeah, it's difficult to read  on the screen. Yeah, all right. I will change here. That's the, yeah, something like this.  Yeah, Chrome is fine. Okay, great. Much better. Thank you. All right. So I will start with these  helper functions that I will talk about in the next minutes. But basically, I wrote these functions  to help in this code demonstration. And I will load the RGAA package. You need to configure  it in your own machine. There are information. You will find this information in the official  website. But in the extra, you will find the dependencies that I installed and what are the  functions that you need to run to set up, for example, the authentication. You just need to  run it one time. And to start the code demo, the code demonstration. First, we need to initialize  the connection with Google Earth Engine. As I said, Google Earth Engine is a cloud computing  platform. So we need to establish a first connection with the platform to start and use it in R.  And I will access my samples that I uploaded in the Google Earth Engine through this function.  So basically, future collection is a representation of vector data in the Google Earth Engine.  And this is the ID, the unique location of some data set that is provided in the Google Earth Engine.  And we need to, the first thing we will run is what is the number of the samples that we have here.  So it's an important part because if you run only this code here, sample size,  you will get only this Earth Engine object number. So basically here, you don't have a result of the  output, the analysis method because in this part of the code, we only have a description of some  type of analysis. So basically, we need to send this description to the platform to run in the  Google Cloud infrastructure and after that download or get the result. You only will have the result  if you call getInfo. So basically, if you want to access any type of result in considering this  method, considering these functions, you need to call the getInfo to get the real result of  analysis forcing the API to access the Google Earth Engine and get the result from the cloud.  So this is one type. We will have the access of the data to the viewer. That is the other type.  So we will access the samples and we will see it in a web map. I will show in the next minutes.  So what are the sample attributes? So basically here, we will see that we have an ID and a proper  name it LandCover containing the values of LandCover for the sample. And in this toy sample, we have  a forest. We have samples for forest, basher land, crops, others, and water. So basically, I will  create a list with one color for each of these classes. And I will use this function that I  implemented to plot all the samples separate then by colors. So basically here, we are seeing the  all the samples organized by layers, by class, and the number of samples that we have in the  each class. And we are seeing, so this is stored in Google Earth Engine and now we are seeing the  results in our studio. So it's some type of the on-the-fly visualization. Basically, this function  is responsible to split the samples and apply the color in the using the parameters of the  Google Earth Engine. What we will do now is calculate and visualize the sample extent. So  you can see that I'm reducing all the geometries of the samples and I'm calculating the bounds, so the  extent of that sample. And I will put this result, this output in a layer and I will put the layer  in a list of layers. And if I send to my function this layers object, it will plot the samples and  the what we have in the list. So basically, we are seeing here the extent of our samples.  So this is a bounding box and it's important because we will use it to refine, to specify what is the  spatial extent for our analysis. So basically, to run this analysis, we will reduce all the images of  Landsat data, considering surface reflectance product, obitained for the whole year of 2019.  I will filter by this tree, by this old spectral bands and I will pass this extent. And this is an  important parameter because this is a mathematical representation of our valid pixels. So Earth  Engineery will use it to remove the cloud and cloud shadow pixels using the Landsat  quality band assurance. So to understand these values, you should access this official documentation  by US GIS to understand what are the values that are maintained here. And I will run a reducer.  So we can access, we have several reducers like standard deviation, median, but in this case,  I'm running the percentile 25, percentiles 50, and the percentile 25. So basically, I will,  so I need to create this. So basically here, we are only, we only have the descriptor. We don't  have all the Landsat data here in my notebook, but we can visualize it in the viewer when we put it  on the map. So for example, if I call this function band names and I get info, so I force it now,  the API to access the real band names in the Google Earth Engineery and retrieve to me  in the object representation here in car. So by default, Google Earth Engineery don't provide  any type of enhance, so automatic visualization for Landsat data. And I create this function to  basically calculate the minimum and the maximum values for each of specified bands. So it's  only a function to define the visualization parameters that will be used in the map. So now  we are seeing, we will see the annual median for all the spectral bands in the, our interesting area.  So we can check, for example, if the forest samples are in the right place and things like that.  So we have all these.  So you calculated these satellite bands, you calculated them in Google Earth Engine, right?  And then you now load it, you display it in R. Is there any like a temporary object that you  download from Google Earth Engine to your machine? I mean, or is it just really loads it in the memory?  How does it work?  No, it's, it's, it's worked really in on the fly here. I only have access to a tile server  that is provided by Google Earth Engineery. So all the analysis were processed in Google Earth Engineery  in here. I only have a visualization of depth, for example, in PNG. And depth is available in the  web map. Okay, so you don't have the actual data? You get the data locally, you just got the PNGs  from the, okay. Yes, okay. But it's important thing because, for example, I can check it and I can see  that that image has a lot of clouds because my, for example, my valid expression, my mathematical  expression actually didn't remove the cloud. So it's good to have this type of representation  before to download the data. Okay, continue, please. So what we will do now, so I, I calculated it for  all these Landsat bands. So it's 21 bands. And now, now I will export the data. So what this function  implements, these, this function extracts the Landsat percentile values for each sample,  export it as a CSV file to Google Drive, download it from Google Drive and load it in R. So basically,  here, I sent to the function the samples and my image. This image is not loaded in the, in the,  it's not in my memory. It's in the Google Earth Engine. So I process all the things in the Google  Earth Engine and I export it on the output as a CSV file. And now I have the real output here in my  memory. So just like a CSV file. So I have all the values for the samples. And basically, I have,  for each spectral band, I have one column. And I can access it right now. If you had, if you had  like a million, million points, and if you had to overlay like thousands of clusters, that could  be computational, right? And yes, yes. And Google block you maybe if you send to too big number,  I mean, how does it work? To, to work with this number of points, my suggestion is don't  use this function, only export the output to Google Drive. So basically, if you look in this  function, this is an exportation task. So basically, you can send like the name of the samples,  the name of feature collection stored in Google Earth Engine, you can send it to the  platform and export is it to Google Drive. So basically, you just need to wait the exportation.  Sometimes we'll take more than one hour, maybe 10 hours. But you, you have, you will have the  result in Google Drive. And after that, you can download it. So basically, now we will filter  this, this data frame to use only the bands that we are interested. So we will have the land cover,  the predicted variable and the, and, and the all the spectral values that was extracted. And  I will convert it to factor to running MLR tree. And basically, I will run my hyper parameter  tuning using MLR tree. There are several options to implement it. I won't talk about too much MLR  tree and you can find more information in this link. But what I will do now is I will run the  optimization and explain the code. So basically, now I have all the samples here in my machine  loaded in or and I will create a classification tasks. It is a basic object in MLR tree. So I'm  sending to the MLR tree my classification matrix and what is the variable, what that I'm  interesting. So my target variable here, I'm using random forest. So it's important to use  random forest because after this hyper parameter optimization, I will classify the entire scene  using Google Earth Engine. So I need to have the same algorithms and the same parameters  implemented in Google Earth Engine because I don't have the Landsat data in my, my local  notebook. So I, I'm only running a hyper parameter optimization here. And your model is fitted in  R and a Google Earth Engine, it doesn't have an R engine. So how do you export the model to  Google Earth Engine? It's not possible. I need to retrain it in Google Earth Engine. Basically,  what, what I will have here is the best hyper parameters for classify random forest for this  specific area. Yes. And the Google Earth Engine doesn't have this functionality of MLR tree to do  the fine tuning, right? So you do the fine, only fine tuning locally, then you get this fine  tuned like a one or two parameters and then you use that in the Google Earth Engine. That's what  you're saying. Okay. Exactly. And so in Google Earth Engine, I can only choose three hyper parameters,  minimal leaf population, number of trees and variable per split, if I consider random forest.  So basically what I'm doing here is finding the best hyper parameters in MLR tree to apply  it in Google Earth Engine, retrain my algorithm and classify the whole, the whole area. I,  I, I, I, I define, define this design. So here we will have all the available parameters that will  be tested in MLR tree. So basically, we can, I think it's more than 2,200 parameters. And  I will show it when the optimization has finished. And I created this tuning instance to define  how it will work. So basically what happens here is the, the tuning will consider five  fold cross validation to maximize the overall occurrence and will test all the values that I  established here in this design. So if I look it, I have all the possibilities of the parameters,  of hyper parameters here. And this function will test all and evaluate it by overall occurrence.  And basically after that, we need to create a tuner object and start the optimization approach.  And the output is presented here. So we can see that we have 86% of overall accuracy considering  all the class. And we have these three values for the hyper parameters. And now we can try one,  try one, it's a bit of a problem. It's, it's too small.  Yes, but, but basically it's compensated by, for example, the number of trees.  Because what happened here, he tested all the combinations and the best overall  accuracy was obtained with that. Of course, this, this could be, for example, this could,  could happen because we have fewer samples. So here we have almost 40, 400 samples.  Probably if you have more samples, it will have different results, but we can visually compare  it with, for example, the full parameter in Google Earth Engine. And we can see if it really  produce a better result. Okay, continue, please. So basically, and other possibility here is we can  see, this is an important feature functionality that doesn't exist in Google Earth Engine. We can  see that what is the best and Landsat metrics that are contributing to the classification approach.  So the best metric is the SWIR 2 percentile 75. So we can see this Hank for all the variables.  And here we have the three optimized values. So basically now we are ready to run the  classification in Google Earth Engine. And what I will do is I will overlay again the points  because it needs to run in Google Earth Engine. So if you check this, this is a function that  overlay inside Google Earth Engine without export it. If you check it, it's just a descriptor.  The whole thing will be processed only when I show something in the map or only when I  call a get info function. So in this case, it will only run on the fly when we see the, we see the  result in the, in the air studio in the map. So basically, I will create two random forest models.  So one using 103 and the default parameters and other using the MLR tree optimized hyper parameters.  And I will train these two models. Again, this is not, this is only a descriptor. The whole thing  will be processed when I put these results in the map. So I'm here, I'm creating a training  order to, to classify the, to create a model. And after that, I will classify the same images  using two different models. And I will put all the things, all these layers inside  Google, inside the, the map to analyze and to see the result.  So I like this visualization here. It's better. So now all, all the processing here are, is,  is running on the fly. So basically we have two classifications that were trained and predicted  in the, this Landsat dataset. And we can compare, visually compare the results. So this is the MLR  tree optimized hyper parameters. And this is the default classification with the full hyper  parameters of Google EarthFansion. And basically, and we can compare it to the real data in using  the Landsat and medium image. So basically we can zoom in in some part and we can check if the result is  what is the difference between the results. So basically we can see more in some places we can  see that they're more consistent. But again, this is only a toy example. You need to create  some different analysis, probably some parts we lost some water, but we have a good convergence,  a good spatial consistent, considering, for example, even high resolution data like this,  for example. So we are seeing the result. And now we can export it. So we have a classification  object and we can export it to Google Drive. And we can start exportation test. It's possible to  export the Landsat medium image. So we can export it to Google Drive too. And you specify, for example,  some last bands, then, for example, all the bands that were calculated. And if we check  in the code editor, basically, I will reload here, all the exportation tasks that were started  in the R we can see here. So it's basically it's running on the Google EarthFansion. And I will  get the result in the Google Drive. So I download the data so you can access the final results in  this folder too. And it's possible to visualize it in the GIS. So yes, this is the this is my code  demonstration. And to summarize, you can try yourself these two possibilities of analysis. So,  for example, you can expand the feature space using more percentile order reducers. And you can use  Sentinel to data in the same area with the same samples to get, for example, different map using  a better data. So to summarize the things, all the points, the use of Google EarthFansion by  scientific community is increasing. So basically, in the last years, we have several publications  in peer review journals that are using Google EarthFansion in different study areas. And we have  from some special issues in peer review journals too. And in my opinion, Google EarthFansion is an  outstanding platform that allows users to test large scale, geospatial analysis quickly and on  the fly. And for me, the best way to analyze to run it to use Google EarthFansion in your workflow  is exporting a final result or intermediate result as we did here. And we you can export  route images, for example, like for all the Landsat images for one year, but it's not recommended  and will take too much time. And you can use multiple Google accounts to operate together  and write in a shared bucket hosted in Google Cloud Storage. So it's an example of that. It's,  for example, a land cover classification for the whole country or the whole continent. You can  have several accounts sending a classification request to write in the same bucket. And it's  possible to use Google EarthFansion to serve images as web GIS application. And you can try to join  to Gel for Good Summit. It's like an annual summit for Google EarthFansion users. And they launch,  for example, the new functionality in this event. So it's a good event to understand better the  platform and what will be the news. And sorry to be a little harsh, but my goal was to present  and to end an analysis and an example to integrate Google EarthFansion with R. All the code is available.  I can respond some emails to if you have other questions later. And now we have time for the  questions. Thank you. Okay, thank you. Okay, we do have time for questions. So I think that was  a very good demo. Giuseppe, we'll start. Please tell the question. I'll repeat. Yeah.  So in Google EarthFansion, can you use random forests for regression problems?  Yes. You have several models available in Google EarthFansion. And you can specify if you run it  for regression or for classification. So this, you can see here, all the, you can see here, all the  in the classifier, you can see all the available models, but you can specify if it will be a  regression, I think it's in the output. So you could specify if it is a classification problem,  a regression problem, or a probability. For example, for binary classifications, you have the  possibility to export the probability for each class. So even though function is called classifying  classifier, it actually can be used for regression problems too. So it's a bit hidden.  Yeah, it's misleading. But yes, it's possible. Yes, it's a pity that, you know, you, because you  do an MLR, you already do the work, right? You do this optimization, and that you cannot then  use exactly that model you got from MLR that you cannot deploy it in Google EarthFansion.  And, and that's because, yeah, I don't know this, is there any intention to make Google EarthFansion,  you know, compatible to more, like more open source solutions or?  No, that's, that's really a good question, because actually, we can't extend Google EarthFansion.  So for example, we can implement some native function here, we only can use it. So did  so the decisions are totally dependent from Google team. So basically, I, I use Google EarthFansion  for a few years, but the whole concept here, it's not like a deployed model, but run the whole team.  Again, so for example, even if I had the, the, the, my model, train it and I save it, I can't  put it in, in Google EarthFansion to run, for example, for other area, I need to retrain the model  and export only the output. So all the team need to run on end to end again. But I know that for  deep learning approach, they have some type of cloud solution named ML cloud, I think, that you can  host, for example, a deep learning, a neural network, train it in TensorFlow in there. And  after that, you can call it from Google EarthFansion. But I think it's just for  a neural network, for other classical machine learning approach, I don't see anything.  Okay. And there was a question, somebody asking Q&A, somebody asked about the bottom neck. So like,  let's say if you, if you have more points or something, then slowly or Google Drive, I think it's  the, on the free account, it's something like 30 gigabytes or something. So then your Google Drive  runs out of space, and then you, you will, you will experience the storage problems, right?  Yes. Actually, if you really want to use Google EarthFansion for a huge analysis and  generate big images, you need to, the best option is use Google Cloud Storage, that you will have  more option, but you can expand your Google Drive account too. But for more production,  to produce a big Earth observation result, you need to run some type of the cloud solution. And  something I didn't mention. So here you have a personal asset. So this is your asset code. So you  have for free, this have this amount of gigabytes available. So for example, any analysis that I  want to only access in Google EarthFansion, I can rewrite it in Google EarthFansion. So basically,  for example, if I don't need this, the output in my local machine, I can rewrite the output  in the Google EarthFansion to access it in a more quickly way, as like, for example,  our regular Landsat image. But in general, you want to, to recover the data and export it from,  from Google EarthFansion to work in other platforms. But it's a, it's a good option,  for example, to split parts of your process. You can process, for example, some NDVI calculation  and generate the median as assets. And after that, you can train using the output of the first analysis.  And so like Microsoft has the cloud services for like R to compute in R to deploy R. And if,  does Google has something like that that you deploy like R and then, and maybe then you could  connect Google Earth Engine and, you know, some R installation, do you think something like this  could work? Yeah, I, I didn't see anything like this, like this. But probably, just a second,  as I think you can help here. One option there is that if you run a server in Google Cloud,  and if you run a P-L on that, you can get these out with your Rintian driver  and you can directly read files from the EarthFansion archive. So you can do your own  processing in your own Rented Service, reading from the Earth Engine archive.  That's all processing is an outside Earth Engine. You don't use Earth Engine processes  if you use the data that's there. GDAL has an Earth Engine driver.  Did you hear that? Did you get that, Landro?  So only the last part that GDAL has an Earth Engine driver. It's right.  Yes. So the idea is that you use the Google Earth Engine functionality in the cloud installation.  Ah, okay. Okay, okay, okay. So from Google Cloud, you can access the raw satellite images.  And so basically, you still do computing inside Google infrastructure, but you don't use the  Google Earth Engine. Yes. But you use the data from the Google Earth Engine. So that's,  that could be a solution, like the backend solution to compute your own way with the  Google Earth Engine data. Okay. No, okay, good. That's good to know. Okay. Are there any more questions?  Giuseppe? Did you hear that? That you can mount the Google Drive?  No, no, I couldn't hear. Sorry. Okay, I will repeat. So Giuseppe said that you can mount the  Google Drive inside your server. And then if you run out of space, so you can move the data,  so you never run out of space. You continuously move the data, so you always keep it within the  limit of 350 gig or something. Yes, yes. That could be a trick. Yeah. Are you cool?  Yeah. With the R clone. Yes, you can export the result, download it and erase. So you  can use only, you can use Google Drive only as temporary storage to get the outputs from  Google Earth Engine. So it could work. And if this package, it's really easy to implement that because  RGAA provides access to Google Drive using an API, the official API. So it could be a good  work to, a good way to implement it. Okay. More questions? Yes, in the back, please speak loudly.  Okay. So there's a ee ee underscore s underscore raster function to export the image  and get it locally, right? And that's in the RGAA package. Yeah. Okay. Okay. So there is  already some wrapper to get the images, but then you depend on your bandwidth and  download the whole world at 30 meters. Just a little chunk. Okay. More questions?  So you mentioned also this visualization. So once you make the analysis, you know,  you've been using the leaflet, but you know, it's useful to create kind of like a web mapping up.  So you put some HTML or something somewhere and then people can access it without needing to run  the code and or, you know, use R. So tell us something about that. How complicated is that?  How much coding do you still have to do? And is that something that you can do then from R or  or you have to use some Google or Tange in? Well, actually, to integrate a Google or  Fange with any type of the type of web GIS solution, you need to create like access token  to get the result for any specific analysis. So for example, for the result of annual and  NDPI or for specific Landsat data, but you need to refresh this token, for example, every day.  So I did that. So you need to create like a publication link through the Python  API and have access to the to our URL and to a token that you provide to you like  tile servers, tile servers to access that analysis. And you can put it in any type of  application like leaflet application or open layers application. But you need to  handle this token. So a good way to do that is republish all the day and refresh that token.  So I think I put here in the there is an application named TD lights. I didn't remember.  But these applications show how integrate Google or Fange any with any type of web applications.  So yes, this one. So mostly you need to publish the layer using Python. And you can access the result  trendy light. So this one. So you can access the output of this analysis analysis to a tile  service API that's that is provided by her fanging. But it's not easy, but it's possible.  Okay. Yeah, we have the on Thursday, we have a team talking about the map view.  So you're going like few lines of code, you create a web map, you know, you customize it and you  you set up things that will be useful to make also for Google or attention to like quickly  create this web mapping apps. But you said you need a token and it's a bit more complicated.  Yeah, but but yeah, thinking better now, I think our other other possibilities like for example,  here, I'm using the RG a package. So I only create this map representation that's probably  is using map review. So you can integrate it with the ecosystem, because all the token creation  and all the steps that I described probably is implemented in this function at layer that is  that is implemented by our GBA. So probably could be option using this output of this function here.  So this is a map view instance to extend the visualizations in the art.
OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_0_623.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_624_1055.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_1056_2687.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_2688_7391.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_7392_8483.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_8484_13103.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_13104_13907.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_13908_16919.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_16920_21815.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_21816_21983.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_21984_23075.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_23076_24323.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_24324_25475.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_25476_30191.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_30192_30455.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_30456_38843.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_38844_40415.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_40416_40583.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_40584_42359.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_42360_44447.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_44448_45203.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_45204_47087.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_47088_47555.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_47556_49223.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_49224_49403.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_49404_51467.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_51468_54107.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_54108_55223.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_55224_55319.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_55320_55547.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_55548_55823.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_55824_56255.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_56256_56339.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_56340_57443.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_57444_57695.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_57696_57887.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_57888_58103.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_58104_58811.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_58812_64247.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_64248_65963.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_65964_66347.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_66348_69263.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_69264_74411.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_74412_74495.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_74496_75515.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_75516_75671.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_75672_75779.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_75780_77471.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_77472_77711.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_77712_80351.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_80352_80999.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_81000_81983.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_81984_82967.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_82968_83087.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_83088_83135.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_83136_83255.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_83256_83711.png OpenGeoHub_Summer_School_2020_Leandro_Leal_Parente_-_GEE_toolbox-_efficient_computing_using_Google_E_83712_85507.png