<jupyter_start><jupyter_text>316 - Californian Fires 2020 - Sentinel-3 SLSTR NRT AOD >> 30 - CASE STUDIES - FIRE PREREQUISITES The following **20 - DATA DISCOVERY** module is a prerequisite:   - [252 - Sentinel-3 SLSTR NRT - Fire Radiative Power - Load and browse](../252_Sentinel-3_SLSTR_NRT_FRP_L2_load_browse.ipynb)It is recommended to go through the module before you start with this module.   3.1.5 Discover Californian Fires 2020   Example Sentinel-3 Near Real Time SLSTR Fire Radiative Power (FRP) The [Copernicus Sentinel-3 (S3) Near Real Time (NRT) Fire Radiative Power (FRP)](https://navigator.eumetsat.int/product/EO:EUM:DAT:0207) product, implemented by EUMETSAT, detects and monitors the **immediate threats of any 'burning' events over an area size of 1 km2**. Commonly gathered under the term of hotspots, these are not only fires over continents, but also flames from persistent land and ocean gas flares, and active erupting volcanoes. It is applicable to **agricultural burning, wildfires, deforestation, tropical peatland fires, industrial gas flares, and volcanoes**, all emit radiative signatures with a high seasonal and geographical variability. The current version of the Near Real Time Sentinel-3 Fire Radiative Power product is considered as 'preliminary operational'. It is mainly applicable during nighttimeGet more product specifications [here](https://www.eumetsat.int/website/home/News/DAT_4962463.html).The following workflow is based on an example of `Sentinel-3 Near Real Time SLSTR FRP` data on 12 September 2020 capturing the severe wildfires in California, US.  Outline* [1 - Load Sentinel-3 NRT SLSTR FRP data](load_s3_frp)* [2 - Load, mask and regrid FRP computed from MWIR channel (3.7 um)](load_mwir)* [3 - Load, mask and regrid FRP computed from SWIR channel (2.25 um)](load_swir)* [4 - Load, mask and regrid FRP computed from SWIR channel (2.25 um) with SAA filter applied](load_swir_nosaa)   Load required libraries<jupyter_code>import xarray as xr

import numpy as np
import matplotlib as mpl
import matplotlib.colors as colors
import matplotlib.cm as cm
import matplotlib.pyplot as plt
import cartopy.crs as ccrs
import cartopy.feature as cfeature
from cartopy.mpl.gridliner import LONGITUDE_FORMATTER, LATITUDE_FORMATTER<jupyter_output><empty_output><jupyter_text>Load helper functions<jupyter_code>%run ../functions.ipynb<jupyter_output><empty_output><jupyter_text>Load Sentinel-3 Near-Real-Time SLSTR FRP data Sentinel-3 Near Real Time SLSTR FRP data are disseminated  in `netCDF` format. The first step is to load the data file with xarray's `open_dataset()` function. Once the data file is loaded, you see that the data file has three dimensions: `columns`, `fires` and `rows`. The data and additional information, such as quality flags or latitude and longitude information, is stored as data variables.There are three variables of interest:- `FRP_MWIR` - Fire Radiative Power computed from MWIR channel (3.7 um) [MW] - to detect fires with temperatures less than 1000 Kelvin- `FRP_SWIR` - Fire Radiative Power computed from SWIR channel (2.25 um) [MW] - to detect fires with temperatures above 1000 Kelvin- `FLAG_SWIR_SAA` - Flag values to filter out South Atlantic ANomalies (SAA) & other transient / spurious events, only applicable to FRP SWIR<jupyter_code>frp_dir = '../eodata/sentinel3/slstr/2020/09/12/'
frp_xr = xr.open_dataset(frp_dir+'FRP_in_California_202000912.nc')
frp_xr<jupyter_output><empty_output><jupyter_text>Load `latitude` and `longitude` information You can already load the `latitude` and `longitude` information, which are two variables in the loaded `xarray.Dataset`. Latitude and longitude information will be required for the regridding process.<jupyter_code>lat_frp = frp_xr['latitude']
lat_frp

lon_frp = frp_xr['longitude']
lat_frp<jupyter_output><empty_output><jupyter_text>Define variables for `plotting` and `gridding` Let us also define some variables for `plotting` and the `gridding` process. For example the sampling size of the gridded FRP values or the geographical extent.<jupyter_code>sampling_lat_FRP_grid = 0.05        # Sampling for gridded FRP values & differenrce  stats computation 
sampling_lon_FRP_grid = 0.05        # Sampling for gridded FRP values & differenrce  stats computation 
FRP_plot_max_grid     = 40.         # Max Integrated FRP value, for plots


lat_min               = 30.        # Minimum latitude for mapping plot [deg N]
lat_max               = 50.        # Maximum latitude for mapping plot [deg N]
lon_min               = -125.        # Minimum lonitude for mapping plot [deg E]
lon_max               = -110.        # Maximum lonitude for mapping plot [deg E]<jupyter_output><empty_output><jupyter_text>Now, let us go through the three different variables (`MWIR`, `SWIR` and `SWIR with SAA filtered out`) and let us load, mask, regrid and visualize them.  Load, mask and regrid `FRP computed from MWIR channel (3.7 um)` The first step is to load the `FRP_MWIR` data variable from the loaded `netCDF` file. You see that in total 983 fire entries were detected.<jupyter_code>frp_mwir = frp_xr['FRP_MWIR']
frp_mwir<jupyter_output><empty_output><jupyter_text>The next step is to extract (mask) only the FRP pixels detected by the `MWIR` channel. Valid pixels are different to -1. You can use the function [generate_masked_array](../functions.ipynbgenerate_masked_array) to extract the right pixels. There were 947 fire entries with a temperature less than 1000 Kelvin detected.<jupyter_code>masked_frp_mwir = generate_masked_array(xarray=frp_mwir, 
                                        mask=frp_mwir, 
                                        threshold=-1.,
                                        operator='!=', drop=True)
masked_frp_mwir<jupyter_output><empty_output><jupyter_text>Let us store the number of hotspots / fires in total and per category as variables.<jupyter_code>n_fire_tot           = len(lat_frp[:])
n_fire_MWIR          = len(masked_frp_mwir.to_masked_array().compressed())

n_fire_tot, n_fire_MWIR<jupyter_output><empty_output><jupyter_text>Generate a gridded FRP array In a next step, let us compute the gridded FRP information. The gridded FRP is the integration (sum) of FRP within a grid cell.You can use the function [slstr_frp_gridding](../functions.ipynbslstr_frp_gridding) for the gridding process.<jupyter_code>FRP_MWIR_grid, lat_grid, lon_grid = slstr_frp_gridding(parameter_array=frp_mwir,
                                                       parameter='mwir',
                                                       lat_min=lat_min, 
                                                       lat_max=lat_max, 
                                                       lon_min=lon_min, 
                                                       lon_max=lon_max, 
                                                       sampling_lat_FRP_grid=sampling_lat_FRP_grid,
                                                       sampling_lon_FRP_grid=sampling_lon_FRP_grid, 
                                                       n_fire=n_fire_MWIR,
                                                       lat_frp=lat_frp,
                                                       lon_frp=lon_frp)

FRP_MWIR_grid, lat_grid, lon_grid<jupyter_output><empty_output><jupyter_text>Mask out the invalid pixels for plotting. You can use numpy's function `np.ma.masked_array()` for this.<jupyter_code>mask_valid = np.where(FRP_MWIR_grid[:,:] != -9999., False, True)
D_mwir = np.ma.masked_array(FRP_MWIR_grid[:,:], mask=mask_valid)
D_mwir<jupyter_output><empty_output><jupyter_text>Calculate some statistics, e.g. mean, std, min and max, and add them to a string that can be integrated in the final plot.<jupyter_code>textstr_1 = 'Total number 1km hot-spots = ' + str(n_fire_MWIR)


FRP_sum = np.sum(masked_frp_mwir.to_masked_array().compressed())
FRP_mean = np.mean(masked_frp_mwir.to_masked_array().compressed())
FRP_std = np.std(masked_frp_mwir.to_masked_array().compressed())
FRP_min = np.min(masked_frp_mwir.to_masked_array().compressed())
FRP_max = np.max(masked_frp_mwir.to_masked_array().compressed())

FRP_sum_str  = '%.1f' % FRP_sum
FRP_mean_str = '%.1f' % FRP_mean
FRP_std_str  = '%.1f' % FRP_std
FRP_min_str  = '%.1f' % FRP_min
FRP_max_str  = '%.1f' % FRP_max

textstr_2 = 'FRP 1 km: \n Total = '+FRP_sum_str+' [MW] \n Avg. = '+ FRP_mean_str + ' [MW] \n Min = ' + FRP_min_str + ' [MW] \n Max = ' + FRP_max_str + ' [MW]'<jupyter_output><empty_output><jupyter_text>Visualize the masked data array with matplotlib's `pcolormesh()` function You can use the function [visualize_s3_frp](../functions.ipynbvisualize_s3_frp) to visualize the Fire Radiative Power computed from the MWIR channels. Additionally, you can take information such as `longname` or `units` from the data variable attributes.The function takes the following keyword arguments:* `data`: DataArray with fire hotspots* `lat`: Latitude information* `lon`: Longitude information* `unit`: Unit of the data* `longname`: Long name of the data* `textstr_1`: Text that contains the total number of 1km hot-spots* `textstr_2`: Text that contains summary statistics of the data* `vmax`: Maximum value to be visualized<jupyter_code>long_name = frp_mwir.long_name
unit = frp_mwir.units
vmax = FRP_plot_max_grid

visualize_s3_frp(data=D_mwir[:,:],
                 lat=lat_grid, 
                 lon=lon_grid, 
                 unit=unit, 
                 longname=long_name, 
                 textstr_1=textstr_1, 
                 textstr_2=textstr_2, 
                 vmax=FRP_plot_max_grid)<jupyter_output><empty_output><jupyter_text>Let us now repeat the processing steps above and let us process and visualize the `Fire Radiative Power computed from the SWIR channel`.  Load, mask and regrid `FRP computed from SWIR channel (2.25 um)` Load the `FRP_SWIR` data variable from the loaded `netCDF` file.<jupyter_code>frp_swir = frp_xr['FRP_SWIR']
frp_swir<jupyter_output><empty_output><jupyter_text>Filter only the pixels that showed a FRP signal from the SWIR channel and filter all pixels equals -1 out. Use the function [generate_masked_array](../functions.ipynbgenerate_masked_array) for the filtering. You see that 312 signals were detected by the SWIR channel, meaning these fires are more severe with having a temperature higher than 1000 Kelvin.<jupyter_code>masked_frp_swir = generate_masked_array(xarray=frp_swir, 
                                        mask=frp_swir, 
                                        threshold=-1., 
                                        operator='!=', 
                                        drop=True)
masked_frp_swir<jupyter_output><empty_output><jupyter_text>Store the number of total fires and per category as variables.<jupyter_code>n_fire_tot           = len(lat_frp[:])
n_fire_SWIR          = len(masked_frp_swir.to_masked_array().compressed())

n_fire_tot, n_fire_SWIR<jupyter_output><empty_output><jupyter_text>Apply the gridding function [slstr_frp_gridding](../functions.ipynbslstr_frp_gridding) and grid the `FRP computed from the SWIR channel`.<jupyter_code>FRP_SWIR_grid, lat_grid, lon_grid = slstr_frp_gridding(parameter_array=frp_swir, 
                                                       parameter='swir', 
                                                       lat_min=lat_min, 
                                                       lat_max=lat_max, 
                                                       lon_min=lon_min, 
                                                       lon_max=lon_max, 
                                                       sampling_lat_FRP_grid=sampling_lat_FRP_grid,
                                                       sampling_lon_FRP_grid=sampling_lon_FRP_grid, 
                                                       n_fire=n_fire_SWIR,
                                                       lat_frp=lat_frp,
                                                       lon_frp=lon_frp,)

FRP_SWIR_grid, lat_grid, lon_grid<jupyter_output><empty_output><jupyter_text>Mask out all invalid pixels for plotting.<jupyter_code>mask_valid = np.where(FRP_SWIR_grid[:,:] != -9999., False, True)
D_swir = np.ma.masked_array(FRP_SWIR_grid[:,:], mask=mask_valid)<jupyter_output><empty_output><jupyter_text>Calculate some statistics, e.g. mean, std, min and max, and add them to a string that can be integrated in the final plot.<jupyter_code>textstr_1 = 'Total number 1km hot-spots = ' + str(n_fire_SWIR)


FRP_sum      = np.sum(masked_frp_swir.to_masked_array().compressed())
FRP_mean     = np.mean(masked_frp_swir.to_masked_array().compressed())
FRP_std      = np.std(masked_frp_swir.to_masked_array().compressed())
FRP_min      = np.min(masked_frp_swir.to_masked_array().compressed())
FRP_max      = np.max(masked_frp_swir.to_masked_array().compressed())

FRP_sum_str  = '%.1f' % FRP_sum
FRP_mean_str = '%.1f' % FRP_mean
FRP_std_str  = '%.1f' % FRP_std
FRP_min_str  = '%.1f' % FRP_min
FRP_max_str  = '%.1f' % FRP_max

textstr_2 = 'FRP 1 km: \n Total = '+FRP_sum_str+' [MW] \n Avg. = '+ FRP_mean_str + ' [MW] \n Min = ' + FRP_min_str + ' [MW] \n Max = ' + FRP_max_str + ' [MW]'<jupyter_output><empty_output><jupyter_text>Apply the plotting function [visualize_s3_frp](../functions.ipynbvisualize_s3_frp) and take some attributes from the attributes stored in the original data file, e.g. `long_name` or `units`.<jupyter_code>long_name = frp_swir.long_name
unit = frp_swir.units
vmax = FRP_plot_max_grid

visualize_s3_frp(data=D_swir[:,:],
                 lat=lat_grid, 
                 lon=lon_grid, 
                 unit=unit, 
                 longname=long_name, 
                 textstr_1=textstr_1, 
                 textstr_2=textstr_2, 
                 vmax=FRP_plot_max_grid)<jupyter_output><empty_output><jupyter_text>Let us repeat the same processing steps for the `FRP computed from SWIR channel`, but with the SAA filter applied.   Load, mask and regrid `FRP computed from SWIR channel (2.25 um) with SAA filter applied` Load the flag for the `South Atlantic Anomaly (SAA)` with the data variable name `FLAG_SWIR_SAA`.<jupyter_code>FLAG_FRP_SWIR_SAA_nc = frp_xr['FLAG_SWIR_SAA']
FLAG_FRP_SWIR_SAA_nc<jupyter_output><empty_output><jupyter_text>Filter out only the fire events detected by the SWIR channel including the South Atlantic Anomaly (SAA) filter. For this, you first have to create a mask including the signals detected by the SWIR channel (`frp_swir` object not equal to -1) and all SAA flags unequal to 0 (FLAG_FRP_SWIR_SAA_nc equals to 0) filtered out.In a further step, you can use the function [generate_masked_array](../functions.ipynbgenerate_masked_array) to apply the mask from the first step to the `frp_swir` `xarray.DataArray`.You see that only 201 fire events fall under this category.<jupyter_code>mask_frp_swir_nosaa = xr.where((frp_swir!=-1.) & (FLAG_FRP_SWIR_SAA_nc==0), 1, 0)
mask_frp_swir_nosaa

masked_frp_swir_nosaa = generate_masked_array(xarray=frp_swir,
                                               mask=mask_frp_swir_nosaa,
                                               threshold=1,
                                               operator='=',
                                               drop=True)
masked_frp_swir_nosaa<jupyter_output><empty_output><jupyter_text>Let us store the total number of fires and of the category `FRP SWIR` with SAA filter applied as variables.<jupyter_code>n_fire_tot           = len(lat_frp[:])
n_fire_SWIR_NoSAA    = len(masked_frp_swir_nosaa.to_masked_array().compressed())

n_fire_tot, n_fire_SWIR_NoSAA<jupyter_output><empty_output><jupyter_text>Apply the gridding function [slstr_frp_gridding](../functions.ipynbslstr_frp_gridding) to grid the filtered `FRP SWIR` data. Additionally to the mandatory kwargs, you have to specify a `flag` kwarg, where you provide the function the `xarray.DataArray` loaded at the beginning, `FLAG_FRP_SWIR_SAA_nc`.<jupyter_code>FRP_SWIR_NoSAA_grid, lat_grid, lon_grid = slstr_frp_gridding(parameter_array=frp_swir,
                                                             parameter='swir_nosaa', 
                                                             lat_min=lat_min, 
                                                             lat_max=lat_max, 
                                                             lon_min=lon_min, 
                                                             lon_max=lon_max, 
                                                             sampling_lat_FRP_grid=sampling_lat_FRP_grid,
                                                             sampling_lon_FRP_grid=sampling_lon_FRP_grid, 
                                                             n_fire=n_fire_SWIR_NoSAA, 
                                                             lat_frp=lat_frp,
                                                             lon_frp=lon_frp,
                                                             flag=FLAG_FRP_SWIR_SAA_nc)

FRP_SWIR_NoSAA_grid, lat_grid, lon_grid<jupyter_output><empty_output><jupyter_text>Mask out the invalid pixels for plotting.<jupyter_code>mask_valid = np.where(FRP_SWIR_NoSAA_grid[:,:] != -9999., False, True)
D_swir_NoSAA = np.ma.masked_array(FRP_SWIR_NoSAA_grid[:,:], mask=mask_valid)<jupyter_output><empty_output><jupyter_text>Calculate some statistics and add them to a string that can be integrated in the final plot.<jupyter_code>textstr_1 = 'Total number 1km hot-spots = ' + str(n_fire_SWIR_NoSAA)


FRP_sum      = np.sum(masked_frp_swir_nosaa.to_masked_array().compressed())
FRP_mean     = np.mean(masked_frp_swir_nosaa.to_masked_array().compressed())
FRP_std      = np.std(masked_frp_swir_nosaa.to_masked_array().compressed())
FRP_min      = np.min(masked_frp_swir_nosaa.to_masked_array().compressed())
FRP_max      = np.max(masked_frp_swir_nosaa.to_masked_array().compressed())

FRP_sum_str  = '%.1f' % FRP_sum
FRP_mean_str = '%.1f' % FRP_mean
FRP_std_str  = '%.1f' % FRP_std
FRP_min_str  = '%.1f' % FRP_min
FRP_max_str  = '%.1f' % FRP_max

textstr_2 = 'FRP 1 km: \n Total = '+FRP_sum_str+' [MW] \n Avg. = '+ FRP_mean_str + ' [MW] \n Min = ' + FRP_min_str + ' [MW] \n Max = ' + FRP_max_str + ' [MW]'<jupyter_output><empty_output><jupyter_text>Apply the function [visualize_s3_frp](../functions.ipynbvisualize_s3_frp) to visualize the data and make use of the data files attributes, e.g. `long_name` or `units`.<jupyter_code>long_name = frp_swir.long_name
unit = frp_swir.units
vmax = FRP_plot_max_grid

visualize_s3_frp(data=D_swir_NoSAA[:,:],
                 lat=lat_grid, 
                 lon=lon_grid, 
                 unit=unit, 
                 longname=long_name + 'no SAA', 
                 textstr_1=textstr_1, 
                 textstr_2=textstr_2, 
                 vmax=FRP_plot_max_grid)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0823936170212766, 'fraction_numerical': 0.011702127659574468, 'mean_word_length': 2.802022244691608, 'pattern_counts': {'":': 0, '<': 105, '<?xml version=': 0, '>': 107, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 77, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '26', 'dir': 'jupyter-structured-clean-dedup', 'id': '200760', 'license': 'MIT', 'n_tokens_mistral': 6203, 'n_tokens_neox': 5871, 'n_words': 1545, 'path': '30_case_studies/315_fire_california_2020_Sentinel-3_SLSTR_NRT_FRP_L2.ipynb', 'repo_name': 'k7t/eo-atmosphere', 'size': '18800'}