<jupyter_start><jupyter_text>Landsat Surface Reflectance * **Products used:** [ls5_sr](https://explorer.digitalearth.africa/ls5_sr),[ls7_sr](https://explorer.digitalearth.africa/ls7_sr),[ls8_sr](https://explorer.digitalearth.africa/ls8_sr)<jupyter_code>**Keywords** :index:`data used; landsat 8`, :index:`data used; landsat 7`, :index:`data used; landsat 5`, :index:`datasets; landsat 8`, :index:`datasets; landsat 7`, :index:`datasets; landsat 5`,<jupyter_output><empty_output><jupyter_text>BackgroundThe United States Geological Survey's (USGS) [Landsat satellite program](https://www.nasa.gov/mission_pages/landsat/overview/index.html) has been capturing images of the African continent for more than 30 years. These data are highly useful for land and coastal mapping studies.DE Africa's Landsat data is ingested from the [USGS Collection 2, Level 2](https://www.usgs.gov/land-resources/nli/landsat/landsat-collection-2?qt-science_support_page_related_con=1qt-science_support_page_related_con) archive and forms a single, cohesive Analysis Ready Data (ARD) package, which allows you to analyse surface reflectance data as-is without the need to apply additional corrections.**Important details:*** Surface reflectance product    * Native scaling range: `1 - 65,455` (`0` is `no-data`)    * To achieve surface reflectance values, normalise values to `0 - 1` using `ds = ds * 2.75e-5 - 0.2`    * Using `dc.load` will load data in the native scaling range `1 - 65,455`, while using `load_ard` will scale the values* [CFMask](https://github.com/USGS-EROS/espa-cloud-masking/tree/master/cfmask) used as cloud mask* Native pixel alignment is `centre`* Date-range: 1984 &ndash; present* Spatial resolution: 30 x 30 mFor a detailed description of DE Africa's Landsat archive, see the DE Africa's [Landsat surface reflectance technical specifications documentation](https://docs.digitalearthafrica.org/en/latest/data_specs/Landsat_C2_SR_specs.html).  DescriptionIn this notebook we will load **Landsat** data using two methods.  Firstly, we will use [dc.load()](../Beginners_guide/03_Loading_data.ipynb) to return a time series of satellite images from a single sensor. Secondly, we will load a time series using the [load_ard()](../Frequently_used_code/Using_load_ard.ipynb) function, which is a wrapper function around the `dc.load` module.  This function will load all the images from Landsat 5,7 & 8, combine them, and then apply a cloud mask. The returned `xarray.Dataset` will contain analysis ready images with the cloudy and invalid pixels masked out.Topics covered include:1. Inspecting the Landsat products and measurements available in the datacube2. Using the native `dc.load()` function to load in Landsat data from a single satellite3. Using the `load_ard()` wrapper function to load in a concatenated, sorted, and cloud masked time series from Landsat 5, 7 & 8.***  Getting startedTo run this analysis, run all the cells in the notebook, starting with the "Load packages" cell.  Load packages<jupyter_code>import datacube

from deafrica_tools.datahandling import load_ard
from deafrica_tools.plotting import rgb<jupyter_output>/env/lib/python3.6/site-packages/geopandas/_compat.py:88: UserWarning: The Shapely GEOS version (3.7.2-CAPI-1.11.0 ) is incompatible with the GEOS version PyGEOS was compiled with (3.9.0-CAPI-1.16.2). Conversions between both will be slow.
  shapely_geos_version, geos_capi_version_string<jupyter_text>Connect to the datacube<jupyter_code>dc = datacube.Datacube(app="Landsat_Surface_Reflectance")<jupyter_output><empty_output><jupyter_text>Available products and measurements List productsWe can use datacube's `list_products` functionality to inspect DE Africa's Landsat products that are available in the datacube. The table below shows the product names that we will use to load the data, a brief description of the data, and the satellite instrument that acquired the data. > We can search for Landsat Collection 2 Surface Reflectance data by using the search term `sr`. `sr` stands for "surface reflectance". The datacube is case-sensitive so this must be typed in lower case.<jupyter_code># List Landsat products available in DE Africa
dc_products = dc.list_products()
display_columns = ['name', 'description']
dc_products[dc_products.name.str.contains(
    'sr').fillna(
        False)][display_columns].set_index('name')<jupyter_output><empty_output><jupyter_text>List measurementsWe can further inspect the data available for each Landsat product using datacube's `list_measurements` functionality. The table below lists each of the measurements available in the data.<jupyter_code>dc_measurements = dc.list_measurements()
dc_measurements.loc['ls8_sr']<jupyter_output><empty_output><jupyter_text>Load Landsat using `dc.load()`Now that we know what products and measurements are available for the products, we can load data from the datacube using `dc.load`.In the example below, we will load data from Landsat 8 from Cape Town for South Africa in January 2018. We will load data from three spectral satellite bands, as well as cloud masking data (`'qa_aerosol'`). By specifying `output_crs='EPSG:6933'` and `resolution=(-30, 30)`, we request that datacube reproject our data to the African Albers coordinate reference system (CRS), with 30 x 30 m pixels. Finally, `group_by='solar_day'` ensures that overlapping images taken within seconds of each other as the satellite passes over are combined into a single time step in the data.> Note: For a more general discussion of how to load data using the datacube, refer to the [Introduction to loading data](../Beginners_guide/03_Loading_data.ipynb) notebook.<jupyter_code># load data
ds = dc.load(product="ls8_sr",
             measurements=['red', 'green', 'blue',
                           'qa_aerosol'],
             output_crs='EPSG:6933',
             y=(-34.31, -34.36),
             x=(18.44, 18.50),
             time=("2018-01", "2018-01"),
             resolution=(-30, 30),
             group_by="solar_day",
             )

print(ds)<jupyter_output><xarray.Dataset>
Dimensions:      (time: 2, x: 194, y: 177)
Coordinates:
  * time         (time) datetime64[ns] 2018-01-14T08:35:40.945747 2018-01-30T...
  * y            (y) float64 -4.126e+06 -4.126e+06 ... -4.131e+06 -4.131e+06
  * x            (x) float64 1.779e+06 1.779e+06 ... 1.785e+06 1.785e+06
    spatial_ref  int32 6933
Data variables:
    red          (time, y, x) uint16 15433 17277 18036 ... 27079 27221 27377
    green        (time, y, x) uint16 13942 16492 17130 ... 26999 27147 27336
    blue         (time, y, x) uint16 12139 14631 15385 ... 28116 28352 28136
    qa_aerosol   (time, y, x) uint8 224 194 224 224 128 ... 192 224 224 224 192
Attributes:
    crs:           EPSG:6933
    grid_mapping:  spatial_ref<jupyter_text>Plotting Landsat dataWe can plot the data we loaded using the `rgb` function. By default, the function will plot data as a true colour image using the 'red', 'green', and 'blue' bands.<jupyter_code>rgb(ds, index=0)<jupyter_output><empty_output><jupyter_text>Load Landsat using `load_ard``load_ard` applies the linear scaling and offset which converts the native `uint16` data to actual surface reflectance values. `load_ard` will additionally concatenate and sort the observations by time, and apply a cloud mask. The result is an analysis-ready dataset.In the example below, we load Landsat 8 data for the same time and place as above. Note a cloud mask has now been applied and the data converted to decimal surface reflectance values.This function will also load images from all the Landsat sensors if they are added to the `products` argument as a list. You can find more information on this function from the [Using load ard](../Frequently_used_code/Using_load_ard.ipynb) notebook.<jupyter_code>ds = load_ard(dc=dc,
              products=["ls8_sr"],
              measurements=['red', 'green', 'blue'],
              output_crs='EPSG:6933',
              y=(-34.31, -34.36),
              x=(18.44, 18.50),
              time=("2018-01", "2018-01"),
              resolution=(-30, 30),
              group_by="solar_day"
             )

print(ds)<jupyter_output>Using pixel quality parameters for USGS Collection 2
Finding datasets
    ls8_sr
Applying pixel quality/cloud mask
Re-scaling Landsat C2 data
Loading 2 time steps
<xarray.Dataset>
Dimensions:      (time: 2, x: 194, y: 177)
Coordinates:
  * time         (time) datetime64[ns] 2018-01-14T08:35:40.945747 2018-01-30T...
  * y            (y) float64 -4.126e+06 -4.126e+06 ... -4.131e+06 -4.131e+06
  * x            (x) float64 1.779e+06 1.779e+06 ... 1.785e+06 1.785e+06
    spatial_ref  int32 6933
Data variables:
    red          (time, y, x) float32 nan nan nan nan nan ... nan nan nan nan
    green        (time, y, x) float32 nan nan nan nan nan ... nan nan nan nan
    blue         (time, y, x) float32 nan nan nan nan nan ... nan nan nan nan
Attributes:
    crs:           EPSG:6933
    grid_mapping:  spatial_ref<jupyter_text>Plot the cloud masked landsat data:<jupyter_code>rgb(ds, index=0)<jupyter_output><empty_output><jupyter_text>*** Additional information**License:** The code in this notebook is licensed under the [Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0). Digital Earth Africa data is licensed under the [Creative Commons by Attribution 4.0](https://creativecommons.org/licenses/by/4.0/) license.**Contact:** If you need assistance, please post a question on the [Open Data Cube Slack channel](http://slack.opendatacube.org/) or on the [GIS Stack Exchange](https://gis.stackexchange.com/questions/ask?tags=open-data-cube) using the `open-data-cube` tag (you can view previously asked questions [here](https://gis.stackexchange.com/questions/tagged/open-data-cube)).If you would like to report an issue with this notebook, you can file one on [Github](https://github.com/digitalearthafrica/deafrica-sandbox-notebooks).**Compatible datacube version:**<jupyter_code>print(datacube.__version__)<jupyter_output>1.8.4.dev52+g07bc51a5<jupyter_text>**Last Tested:**<jupyter_code>from datetime import datetime
datetime.today().strftime('%Y-%m-%d')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10097239956782242, 'fraction_numerical': 0.05205775464099794, 'mean_word_length': 4.2268993839835725, 'pattern_counts': {'":': 0, '<': 43, '<?xml version=': 0, '>': 45, 'https://': 12, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '152681', 'license': 'Apache-2.0', 'n_tokens_mistral': 3498, 'n_tokens_neox': 3076, 'n_words': 1086, 'path': 'Datasets/Landsat_Surface_Reflectance.ipynb', 'repo_name': 'FlexiGroBots-H2020/deafrica-sandbox-notebooks', 'size': '10181'}