<jupyter_start><jupyter_text>Google Earth Engine - Climate data accessThere are multiple climate datasets avaialble in the Google Earth Engine (GEE) data catalog. In this example, we will look at two of these datasets, both of which are produced by NASA. You can visit https://developers.google.com/earth-engine/datasets/tags/climate to check out other datasets in the catalog that are tagged "climate".This tutorial will walk users through accessing GEE data via the Python API, aggregating the raw data to a different temporal resolution, displaying data via maps/charts, and exporting data from GEE to be used locally.The code used in the tutorial is also available in javascript via the code editor:*   NLDAS-2 Script -- https://code.earthengine.google.com/a0313d92d3dd8c01426af856b61c604f?noload=true*   DAYMET Script -- https://code.earthengine.google.com/403bac7e0ba61e24d0dfedcf5f0d52b3?noload=true  Import and authenticate Google Earth EngineGoogle Earth Engine in Colab depends on the Earth Engine Pyton API. This is installed already with Colab, so all we need to do is import and authenticate to get started. After running the ee.Initialize() command you will need to follow the prompts to verify your GEE account.<jupyter_code>import ee
# Trigger the authentication flow.
ee.Authenticate()

# Initialize the library.
ee.Initialize()<jupyter_output>To authorize access needed by Earth Engine, open the following URL in a web browser and follow the instructions. If the web browser does not start automatically, please manually browse the URL below.

    https://accounts.google.com/o/oauth2/auth?client_id=517222506229-vsmmajv00ul0bs7p89v5m89qs8eb9359.apps.googleusercontent.com&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fearthengine+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdevstorage.full_control&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&response_type=code&code_challenge=kP6Q8FWsDOT-xF9dlLGxgOWGsvUH2y2DjJqMJu-RjlI&code_challenge_method=S256

The authorization workflow will generate a code, which you should paste in the box below. 
Enter verification code: 4/1AX4XfWhsnELuJGDh0lBzh_fGTZY7Ly2sQ7vU954mYzxqcQPXMtNazorujp0

Successfully saved authorization token.<jupyter_text>NASA Climate Data in Google Earth Engine NLDAS-2The first dataset we will explore is the North American Land Data Assimilation System Forcing Fields (NLDAS-2). The NLDAS-2 contains a variety of hourly variables at a ~14km spatial resolution from 1979 - present. See the full dataset description here: https://developers.google.com/earth-engine/datasets/catalog/NASA_NLDAS_FORA0125_H002 DAYMETThe second dataset we will use in this notebook is the Daily Surface Weather and Climatological Summaries (DAYMET). From 1980 - 2020, DAYMET contains daily summaries of climate variables at a 1000m spatial resolution. https://developers.google.com/earth-engine/datasets/catalog/NASA_ORNL_DAYMET_V4  NLDAS-2 data -- hourly to dailyIn this example, we will take the raw hourly data and aggregate it for each day. To start, we point to the data in GEE. It is useful to print the names of the bands in the NLDAS-2 image collection so we know which variables are available. Anytime you want to print something from GEE, you need to use getInfo(), which essentially retrieves the object from the GEE server (server-side) and brings it to this notebook (client-side) in a format that can be interpreted.<jupyter_code># point to the NLDAS-2 image collection in GEE
nldas = ee.ImageCollection('NASA/NLDAS/FORA0125_H002')

# create a list of the bandnames in the first image in the collection
bn = nldas.first().bandNames()

# use getInfo() to pull the list of bandnames from the GEE backend to our notebook
bn.getInfo()<jupyter_output><empty_output><jupyter_text>In order to aggregate the NLDAS-2, we need to define a range of dates for which we want to compile daily data. We then need to turn that range into a list so that we can map a function over that list to get daily images for the climate variable of interest.In this case, we will use the month of May 2017 as our time range.<jupyter_code>#list of dates from start to end to map over
start = "2017-05-01"
end = "2017-05-31"
diff = ee.Date(end).difference(ee.Date(start), "day") ## get the difference between the start and end dates

def addDay(day):
  return ee.Date(start).advance(day, "day") ## this function just adds one day to the 

listdates = ee.List.sequence(0,diff).map(addDay)
listdates.getInfo()<jupyter_output><empty_output><jupyter_text>As you can see above, printing out the list of dates using getInfo() is not very informative since the dates are formatted as unix/epoch time. This section reformats those into a format that is easier to interpret. This is not necessary for aggregating the data but is a nice way to check that the date list function worked as intended.<jupyter_code>#reformat dates

def dateForm(date):
  return ee.Date(date).format("YYYY-MM-dd")
  
ymdDates = listdates.map(dateForm)
ymdDates.getInfo()<jupyter_output><empty_output><jupyter_text>The next section is where all the computation occurs. It starts by defining the function that will do all the work of aggregating the hourly climate variables to daily averages. So for each day, the function returns an image that is the daily average of whichever climate variable we choose, in this case "temperature".Once the function is defined, it is mapped over the list of days we created and the output is a collection of images, one for each day in our list.<jupyter_code>def meanDaily(day):
  return ee.ImageCollection(nldas).select("temperature").filter(ee.Filter.date(ee.Date(day), ee.Date(day).advance(1, "day"))).mean().set({"system:time_start": day})

temperature = ee.ImageCollection(ymdDates.map(meanDaily))

temperature.size().getInfo()<jupyter_output><empty_output><jupyter_text>We can use folium to visualze the result. This chunk adds the first image in our collection to the map. Average Daily temperature for May 1, 2017.<jupyter_code># Import the Folium library.
import folium

# Define a method for displaying Earth Engine image tiles to folium map.
def add_ee_layer(self, ee_image_object, vis_params, name):
  map_id_dict = ee.Image(ee_image_object).getMapId(vis_params)
  folium.raster_layers.TileLayer(
    tiles = map_id_dict['tile_fetcher'].url_format,
    attr = 'Map Data &copy; <a href="https://earthengine.google.com/">Google Earth Engine</a>',
    name = name,
    overlay = True,
    control = True
  ).add_to(self)

# Add EE drawing method to folium.
folium.Map.add_ee_layer = add_ee_layer

# Set visualization parameters.
vis_params = {
  'min': -5.0,
  'max': 40.0,
  'palette': ['3d2bd8', '4e86da', '62c7d8', '91ed90', 'e4f178', 'ed6a4c'],
};

# Create a folium map object.
my_map = folium.Map(location=[30, -90], zoom_start=4)

# Add the elevation model to the map object.
my_map.add_ee_layer(temperature.first(), vis_params, 'Mean Daily Temperature - May 1 2017 - NLDAS')

# Add a layer control panel to the map.
my_map.add_child(folium.LayerControl())

# Display the map.
display(my_map)<jupyter_output><empty_output><jupyter_text>DAYMET data -- daily to monthlyIn this example, we will take the raw daily data frm DAYMET and aggregate it for each month.<jupyter_code># point to the DAYMET image collection in GEE and filter it to the time period of interest
start = "2017-01-01"
end = "2018-01-01"
daymet = ee.ImageCollection('NASA/ORNL/DAYMET_V4').filter(ee.Filter.date(start,end))

# creat a list of the bandnames in the first image in the collection
bn = daymet.first().bandNames()

# use getInfo() to pull the list of bandnames from the GEE backend to our notebook
bn.getInfo()<jupyter_output><empty_output><jupyter_text>Because in this example, we are aggregating the daily data for each month in the range, we need a list of months to map our function over. Since we are using a single year as our time period, we can generate a list of numbers from 1-12 for each month in the year.<jupyter_code>months = ee.List.sequence(1,12);<jupyter_output><empty_output><jupyter_text>Next we need to write the function that will aggregate the data for us.<jupyter_code>def byMonth(m):
  means = daymet.filter(ee.Filter.calendarRange(m, m, 'month')).select(['tmin', 'tmax']).mean() # aggregate the temp bands by mean
  sums = daymet.filter(ee.Filter.calendarRange(m, m, 'month')).select('prcp').sum() # aggregate the precip band to get monthly total
  return means.addBands(sums).set('month',m)

daymetMonth = ee.ImageCollection(months.map(byMonth)) # map the function over every item in our month list

print(daymetMonth.size().getInfo()) # should be 12 images in the collection<jupyter_output>12<jupyter_text>Now we can display the image on a map. In this case, we choose to display 3 images for May (month = 5):1.   Total Precipitation2.   Minimum Temperature3.   Maximum Temperature<jupyter_code># Set visualization parameters for precip
precVis = {
  'min': 0,
  'max': 500,
  'palette': ['ed6a4c','e4f178','91ed90','62c7d8','4e86da','3d2bd8'],
};

# Set visualization parameters for temp
tempVis = {
  'min': -5.0,
  'max': 25.0,
  'palette': ['3d2bd8', '4e86da', '62c7d8', '91ed90', 'e4f178', 'ed6a4c'],
};

# Create a folium map object.
daymetMap = folium.Map(location=[30, -90], zoom_start=4)

# choose image to display
prcpTot = daymetMonth.filter(ee.Filter.eq('month', 5)).select('prcp').first()
tmpMin = daymetMonth.filter(ee.Filter.eq('month', 5)).select('tmin').first()
tmpMax = daymetMonth.filter(ee.Filter.eq('month', 5)).select('tmax').first()

# Add the elevation model to the map object.
daymetMap.add_ee_layer(prcpTot, precVis, 'Total Precip - May 2017 - DAYMET')
daymetMap.add_ee_layer(tmpMin, tempVis, 'Minimum Temperature - May 2017 - DAYMET')
daymetMap.add_ee_layer(tmpMax, tempVis, 'Maximum Temperature - May 2017 - DAYMET')

# Add a layer control panel to the map.
daymetMap.add_child(folium.LayerControl())

# Display the map.
display(daymetMap)<jupyter_output><empty_output><jupyter_text>Detecting Extreme Events - DAYMETWhat if we were interested in detecting abnormal or extreme events, with regard to temperature or precipitation for every pixel? The simplest way would be to just aggregate the daily data using a maximum rather than a mean or a sum. Doing this would resuly in an image collection where each pixel represents the daily maximum of the chosen climate variable for every month. The block below illustrates how you would edit our aggregating function to include the maximum of the tmax parameter (daily high temperature) and precipitation.Because we now have multiple bands with the same name, it is a good idea to rename them. This will help us keep track of which band is which as we aggregate the same band multiple ways.<jupyter_code>def byMonth(m):
  means = daymet.filter(ee.Filter.calendarRange(m, m, 'month')).select(['tmin', 'tmax']).mean().rename(['tmin_avg','tmax_avg']) # aggregate the temp bands to get daily average for each month
  sums = daymet.filter(ee.Filter.calendarRange(m, m, 'month')).select('prcp').sum().rename('prcp_total') # aggregate the precip bands to get monthly total
  max = daymet.filter(ee.Filter.calendarRange(m, m, 'month')).select(['tmax','prcp']).max().rename('tmax_max','prcp_max') # aggregate the temp and precip bands to get daily maximum

  return means.addBands(sums).addBands(max).set('month',m)

daymetMonth = ee.ImageCollection(months.map(byMonth)) # map the function over every item in our month list

print(daymetMonth.size().getInfo()) # should be 12 images in the collection
print(daymetMonth.first().bandNames().getInfo())<jupyter_output>12
['tmin_avg', 'tmax_avg', 'prcp_total', 'tmax_max', 'prcp_max']<jupyter_text>Another idea might be to use the newly created daily max values for each pixel to calculate something else that might indicate an extreme event. The function could be adapted further to include the proportion of the mean represented by the maximum value. For precipitation, if this value were 1, it would mean that the pixel got all of its precipitation for the month on a single day.For temperature, it might make sense to calculate how many degrees above the mean the maximum daily high temperature is for each pixel.<jupyter_code>def byMonth(m):
  means = daymet.filter(ee.Filter.calendarRange(m, m, 'month')).select(['tmin', 'tmax']).mean().rename(['tmin_avg','tmax_avg']) # aggregate the temp bands to get daily average for each month
  sums = daymet.filter(ee.Filter.calendarRange(m, m, 'month')).select('prcp').sum().rename('prcp_total') # aggregate the precip bands to get monthly total
  max = daymet.filter(ee.Filter.calendarRange(m, m, 'month')).select(['tmax','prcp']).max().rename('tmax_max','prcp_max') # aggregate the temp and precip bands to get daily maximum
  extremeTemp = max.select('tmax_max').subtract(means.select('tmax_avg')).rename('temp_dmean')
  extremePrec = max.select('prcp_max').divide(sums).rename('prcp_perc_total')

  return means.addBands(sums).addBands([extremeTemp, extremePrec]).set('month',m)

daymetMonth = ee.ImageCollection(months.map(byMonth)) # map the function over every item in our month list

print(daymetMonth.size().getInfo()) # should be 12 images in the collection
print(daymetMonth.first().bandNames().getInfo())<jupyter_output>12
['tmin_avg', 'tmax_avg', 'prcp_total', 'temp_dmean', 'prcp_perc_total']<jupyter_text>Exporting DataData generated in GEE can be exported in a number of different formats. If you are interested in just the values at certain locations you can export data as a CSV file. In this example, we will demonstrate how to extract a raster image for a given region of interest that can be used as an input in a species distribution model. Export DAYMET montly data for the western USWe will use the 2018 TIGER dataset to define our region of interest. The dataset can be filtered by the name of the state or states of interest. Filtering collections using a list is more efficient than stringing together multiple filters for the same metadata property.We then need to turn those feature (shape + attributes) into just a geometry object (shape outline only). The .bounds() call is not necessary but it creates a box that represents the maximum bounding geometry of the selected states. In this case, that maximum bounding geometry ends up encompassing all of the western states, with partial coverage of ND, SD, NE and TX.<jupyter_code>tiger = ee.FeatureCollection("TIGER/2018/States")

stNames = ["Montana","California", "Washington","Oregon","Colorado","New Mexico"]

roi = tiger.filter(ee.Filter.inList("NAME", stNames)).geometry().bounds()<jupyter_output><empty_output><jupyter_text>Print some info about the image collectionAs a reminder, our image collection contains 12 images, one for each month of the year in 2017. There are 5 bands in the image representing the variables we computed from the raw daily data and each image in the collection has an identifier (number) for the month in it's metadata.<jupyter_code>print('size of collection:', daymetMonth.size().getInfo()) # should be 12 images in the collection
print('bands in image:',daymetMonth.first().bandNames().getInfo())
print('month ids:', daymetMonth.aggregate_array('month').getInfo())<jupyter_output>size of collection: 12
bands in image: ['tmin_avg', 'tmax_avg', 'prcp_total', 'temp_dmean', 'prcp_perc_total']
month ids: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]<jupyter_text>Select a single image to exportEarth engine is set up to export images, not image collections. For smaller collections it is possible to create loops that will export all images in a collection. There is also a module from <NAME> that allows users to export image collections: https://github.com/fitoprincipe/geetools-code-editor/wiki/Batch.In this example, we will select an individual image from our collection and export it. Two things to note when selecting an image for export:1.   Even though you may filter the image collection to select just one image, GEE will still interpretted it as an image collection of size 1. The .first() command below translates the collection to a single image.2.   When exporting and image with multiple bands, all bands must be of the same data type. We use .toFloat() below to ensure all bands are converted to float prior to export.<jupyter_code>imageToDownload = daymetMonth.filter(ee.Filter.eq('month',5)).first().toFloat() # select the image for May 2017<jupyter_output><empty_output><jupyter_text>Export the selected imageWith the image selected, we can create a task to run. Images can be exported to the drive (associated with your GEE login) like in the example below, to google cloud storage, or as an asset in your GEE account. See this guide for more details on exporting from the code editor: https://developers.google.com/earth-engine/guides/exporting.<jupyter_code>task = ee.batch.Export.image.toDrive(image=imageToDownload, # create task
                                     description='daymet_vars_May_2017', # description that will appear in the GEE task manager
                                     scale=1000, # native scale of daymet data
                                     region=roi, # roi object we defined above
                                     fileNamePrefix='2017_05_daymet_westUS', # name of the file in google drive
                                     crs='EPSG:4326', # coordinate reference for output image
                                     fileFormat='GeoTIFF')
task.start() # run the task object<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0898497743759639, 'fraction_numerical': 0.021248643399782944, 'mean_word_length': 5.024776324845148, 'pattern_counts': {'":': 1, '<': 63, '<?xml version=': 0, '>': 63, 'https://': 9, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '453104', 'license': 'MIT', 'n_tokens_mistral': 5437, 'n_tokens_neox': 5099, 'n_words': 2025, 'path': 'geeclimate.ipynb', 'repo_name': 'ryan-p-rock/nasa-twsc-demos', 'size': '17518'}