<jupyter_start><jupyter_text>Stratified sampling, filter and binarizeThe following notebook does a few extra things. We use a global permanent water dataset and create points for stratified sampling over permanent water and land. The notebook then introduces Normalized Difference Water Index (MNDWI) and the stratified sampling value to get a estimated threshold to MNDWI. This is translated as a binary land water image and added back.<jupyter_code>import bqplot
import datetime
import dateutil.parser
import ee
import ipywidgets
import ipyleaflet
import IPython.display
import numpy as np
import pprint
import pandas as pd
import traitlets

# Configure the pretty printing output & initialize earthengine.
pp = pprint.PrettyPrinter(depth=4)
ee.Initialize()
# Function to get sizes in Human readable format
suffixes = ['B', 'KB', 'MB', 'GB', 'TB', 'PB']
def humansize(nbytes):
    i = 0
    while nbytes >= 1024 and i < len(suffixes)-1:
        nbytes /= 1024.
        i += 1
    f = ('%.2f' % nbytes).rstrip('0').rstrip('.')
    return '%s %s' % (f, suffixes[i])<jupyter_output><empty_output><jupyter_text>*While single images are great to do quick analytics, the true power of the Earth Engine environment comes with the possibility of looking at really large and heavy image collections. In GEE environment image collections have their own characteristic setup and are often composed of multiple single images. They often have the similar band structure and generally share a similar metadata structure for filtering and querying.*In this step we query an image collection and get the size and numbe of items in a collection.<jupyter_code>#Public Image Collections
l8sr = ee.ImageCollection('LANDSAT/LC08/C01/T1_SR').filterDate('2018-08-01','2018-12-01').filterMetadata('CLOUD_COVER','less_than',2)
global_water = (ee.ImageCollection('JRC/GSW1_0/YearlyHistory')
                .filterDate('2015-01-01','2018-01-01'))

#Select a single grid to be used for the analysis
grid=ee.FeatureCollection('users/samapriya/vector/la-subset').filterMetadata('Grids','equals',"Grid_1")
                
# Get collection size
print('Total number of JRC Yearly Water History assets with filters: '+str(global_water.size().getInfo()))
# Get sample image from collection
pp.pprint(global_water.first().getInfo())
band_names_original = global_water.first().bandNames()
band_names_original.getInfo()
# Function to get tilelayer url from earthengine server
def GetTileLayerUrl(ee_image_object):
  map_id = ee.Image(ee_image_object).getMapId()
  tile_url_template = "https://earthengine.googleapis.com/map/{mapid}/{{z}}/{{x}}/{{y}}?token={token}"
  return tile_url_template.format(**map_id)
#Create a slider widget to add both Landsat 8 and PlanetScope 4B SR imagery
map1 = ipyleaflet.Map(
    center=(29.2963,-90.3446), zoom=12,
    layout={'height':'500px'},
)

#Definte mask for not water
global_nw=global_water.median().updateMask(global_water.median().eq(0))
jrc_tile_url=GetTileLayerUrl(global_water.median().visualize(min=0, max=3, bands=['waterClass'],palette='grey,brown,FFFFFF,blue'))
l8sr_tile_url = GetTileLayerUrl(l8sr.median().visualize(min=100, max=3500, gamma=1.5, bands= ['B5', 'B3', 'B2']))  #Landsat 8 SR
left = ipyleaflet.TileLayer(url=jrc_tile_url)
right=ipyleaflet.TileLayer(url=l8sr_tile_url)
control = ipyleaflet.SplitMapControl(left_layer=left, right_layer=right)
map1.add_control(control)
map1<jupyter_output><empty_output><jupyter_text>Stratified Sampling The JRC land water mask has a single band called **waterClass** which has the following classification* 0 nodata* 1 not water* 2 seasonal water* 3 permanent waterIn the following setup we are going to do use startified sampling to select permanent land and water. These points are then used to collect the reflectance values for each of these points as a table.<jupyter_code>#Select permanent water and not water
water=global_water.first().clip(grid).gte(3)
land=global_water.first().clip(grid).eq(1)

def geomft(ft):
    ftgeom=ft.setGeometry(ee.Geometry.Point([ft.get('longitude'), ft.get('latitude')]))
    return ftgeom

def strat(image,numpoints):
    img = image.updateMask(image).addBands(ee.Image.pixelLonLat()).stratifiedSample(
    numPoints=numpoints,
    classBand='waterClass',
    seed=1,
    projection='EPSG:4326',
    scale=30,
    region= grid)
    return img.map(geomft)

stratified_water=strat(image=water,numpoints=50)
stratified_land=strat(image=land,numpoints=50)

# print(stratified_water.getInfo())
# print(stratified_land.getInfo())

# Get Landsat 8 SR band values for each class type from Sampling
landMean = l8sr.filterBounds(grid).median().reduceRegion(
  reducer= ee.Reducer.mean(), 
  geometry= stratified_land, 
  scale= 30
  ).values()

waterMean = l8sr.filterBounds(grid).median().reduceRegion(
  reducer= ee.Reducer.mean(), 
  geometry= stratified_water, 
  scale= 30
  ).values()

print('Land Mean for Bands:'+'\n'+str(l8sr.first().bandNames().getInfo()))
print(landMean.getInfo())

print('\n'+'water Mean for Bands:'+'\n'+ str(l8sr.first().bandNames().getInfo()))
print(waterMean.getInfo())<jupyter_output>Land Mean for Bands:
['B1', 'B2', 'B3', 'B4', 'B5', 'B6', 'B7', 'B10', 'B11', 'sr_aerosol', 'pixel_qa', 'radsat_qa']
[230.12, 2962.62, 2942.94, 267.94, 402.78, 397.42, 1023.04, 838.32, 471.16, 322.0, 0.0, 97.2]

water Mean for Bands:
['B1', 'B2', 'B3', 'B4', 'B5', 'B6', 'B7', 'B10', 'B11', 'sr_aerosol', 'pixel_qa', 'radsat_qa']
[174.96, 2953.84, 2934.94, 197.46, 298.24, 242.1, 63.08, 25.86, 17.48, 324.0, 0.0, 68.0]<jupyter_text>Creating a Binary using an Index Approach The modified normalized difference water index ($MNDWI$) is a band ratio that is related to the amount of percentage water that is present in a pixel. For Landsat 8 the setup would be\begin{equation*}MNDWI = \frac{GREEN — SWIR}{GREEN + SWIR} = \frac{Band3 — Band6}{Band3 + Band6}\end{equation*}where $SWIR$ is the short wave infrared band and $Green$ is green band.<jupyter_code>#Create a slider widget to add both Landsat 8 and Landsat NDVI imagery
img = l8sr.median()
ndwi = img.normalizedDifference(['B3','B6']).rename('ndwi')

map2 = ipyleaflet.Map(
    center=(29.2963,-90.3446), zoom=12,
    layout={'height':'500px'},
)
l8sr_tile_url = GetTileLayerUrl(l8sr.median().visualize(min=100, max=3500, gamma=1.5, bands= ['B5', 'B3', 'B2']))  #Landsat 8 SR
ndwi_tile_url = GetTileLayerUrl(ndwi.visualize(bands=['ndwi'], min=-0.2, max=0.6, palette='brown,9999ff,blue'))  #Landsat 8 SR
left = ipyleaflet.TileLayer(url=l8sr_tile_url)
right=ipyleaflet.TileLayer(url=ndwi_tile_url)
control = ipyleaflet.SplitMapControl(left_layer=left, right_layer=right)
map2.add_control(control)
map2
#Get min max region for the stratified sampling points for ndwi over land
ndviland = ee.Image(ndwi).reduceRegion(
  reducer= ee.Reducer.minMax(), 
  geometry= stratified_land, 
  scale= 30
  ).values()

ndviwater = ee.Image(ndwi).reduceRegion(
  reducer= ee.Reducer.minMax(), 
  geometry= stratified_water, 
  scale= 30
  ).values()

#pp.pprint(zonal.getInfo())
print('Max Min NDVI for Stratified Sampling of Land '+str(ndviland.getInfo()))
print('Max Min NDVI for Stratified Sampling of Water '+str(ndviwater.getInfo()))
# From the earlier min max we find that we can safely assume that NDVI >0 equals land (assumption)
constrained= ee.Image(ndwi).clip(grid).gt(0.2)
print(constrained.getInfo())
#Create a slider widget to add both Landsat 8 and Landsat NDVI imagery
img = l8sr.median()

map3 = ipyleaflet.Map(
    center=(29.2963,-90.3446), zoom=12,
    layout={'height':'500px'},
)
l8sr_tile_url = GetTileLayerUrl(l8sr.median().clip(grid).visualize(min=100, max=3500, gamma=1.5, bands= ['B5', 'B3', 'B2']))  #Landsat 8 SR
constrained_tile_url = GetTileLayerUrl(constrained.visualize(bands=['ndwi'],palette='brown,blue'))  #NDWI constrained
left = ipyleaflet.TileLayer(url=l8sr_tile_url)
right=ipyleaflet.TileLayer(url=constrained_tile_url)
control = ipyleaflet.SplitMapControl(left_layer=left, right_layer=right)
map3.add_control(control)
map3<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13296019900497513, 'fraction_numerical': 0.04390547263681592, 'mean_word_length': 5.768518518518518, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 18, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '649974', 'license': 'Apache-2.0', 'n_tokens_mistral': 3013, 'n_tokens_neox': 2734, 'n_words': 732, 'path': 'GEE/2. Stratified Sampling.ipynb', 'repo_name': 'samapriya/training-workshop', 'size': '8040'}