<commit_before>---
title: Landsat ETM+ to OLI Harmonization
description: Inter-sensor Landsat harmonization and time series visualization application.
author: jdbcode
tags: landsat, etm+, oli, tm, harmonization, time-series, join, reduce, visualization, fmask, cloud-mask, analysis-ready-data, ard
date_published: 2019-08-16
---
<!--
Copyright 2019 Google LLC

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->

[Open In Code Editor](https://code.earthengine.google.com/f06171a7bffc0a926d7e802b1f0580eb)

# Landsat ETM+ to OLI Harmonization

This tutorial concerns harmonizing Landsat ETM+ surface reflectance to Landsat OLI
surface reflectance and vice versa. It provides:

- A spectral translation function
- Functions to create analysis ready data
- A time series visualization example

The objective is to provide an end-to-end guide for visualizing a 35+ year regional
time series of Landsat data that can be immediately applied to your own region(s)
of interest.

Note that the characteristics of Landsat ETM+ and Landsat TM data are assumed equal
in this tutorial. As such, with regard to harmonization transformations, reference
to ETM+ is synonymous with TM.

## Context

[Landsat](https://landsat.gsfc.nasa.gov/about/) is a satellite imaging program
that has been collecting moderate resolution Earth imagery since 1972.
As the longest space-based Earth observation system, it provides a valuable
temporal record for identifying spatiotemporal trends in landscape change.

## Why harmonization?

[Roy et al. 2016](https://openprairie.sdstate.edu/cgi/viewcontent.cgi?referer=https://scholar.google.com/&httpsredir=1&article=1035&context=gsce_pubs)
demonstrate that there are small, but potentially significant differences between
the spectral characteristics of Landsat ETM+ and OLI, depending on application.
Reasons you might want to harmonize datasets include producing a long time series
that spans Landsat TM, ETM+, and OLI or you want to produce near-date intra-annual
composites to reduce the effects of missing observations from ETM+ SLC-off gaps
and masking clouds/shadows. Please see the linked manuscript above for more information.

## Instructions

### Functions

Define all the functions first.

#### Harmonization

Harmonization is achieved via linear translation of ETM+ spectral space to OLI
spectral space according to coefficients presented in
Roy et. al. (2016) Table 2 OLS regression. The following snippet defines a dictionary
containing slope (`slopes`) and intercept (`itcps`) images with respective slope
and intercept values per reflectance band.

```js
var coefficients = {
  itcps: ee.Image.constant([0.0003, 0.0088, 0.0061, 0.0412, 0.0254, 0.0172]).multiply(10000),
  slopes: ee.Image.constant([0.8474, 0.8483, 0.9047, 0.8462, 0.8937, 0.9071])
};
```

Band names for the same spectral response window vary between ETM+ (TM) and OLI
and should be unified. The following function selects only reflectance bands and
the `pixel_qa` bands from any given Landsat surface reflectance image and forces
all band names to be consistent with OLI.

```js
function oliBands(img) {
  return ee.Image(ee.Algorithms.If(
    ee.Algorithms.IsEqual(img.get('SATELLITE'), ee.String('LANDSAT_8')),
    img.select(['B2', 'B3', 'B4', 'B5', 'B6', 'B7', 'pixel_qa']),
    img.select(['B1', 'B2', 'B3', 'B4', 'B5', 'B7', 'pixel_qa'], ['B2', 'B3', 'B4', 'B5', 'B6', 'B7', 'pixel_qa'])
  ));
}
```

Finally, define the translation function, which applied the linear model, casts
type as Int16 for consistency with OLI, and reattaches the `pixel_qa` bands for
later use in cloud and shadow masking.

```js
function etm2oli(img) {
  return img.select(['B2', 'B3', 'B4', 'B5', 'B6', 'B7'])
    .multiply(coefficients.slopes)
    .add(coefficients.itcps)
    .round()
    .toShort()
    .addBands(img.select('pixel_qa'));
}
```

#### Cloud and shadow masking

Analysis ready data should have clouds and cloud shadows masked out. The following
function uses the CFmask ([Zhu et al, 2015](https://www.sciencedirect.com/science/article/abs/pii/S0034425714005069))
`pixel_qa` band included with each Landsat USGS surface reflectance image to set
pixels identified as cloud and cloud shadow to null.

```js
function fmask(img) {
  var cloudShadowBitMask = 1 << 3;
  var cloudsBitMask = 1 << 5;
  var qa = img.select('pixel_qa');
  var mask = qa.bitwiseAnd(cloudShadowBitMask).eq(0)
    .and(qa.bitwiseAnd(cloudsBitMask).eq(0));
  return img.updateMask(mask);
}
```

#### Spectral index calculation

Define a function to calculate normalized burn ratio (NBR). The following
application example looks at the spectral history of a forested pixel experiencing
a wildfire. NBR is used here because
[Cohen et al. (2018)](https://www.fs.fed.us/rm/pubs_journals/2018/rmrs_2018_cohen_w001.pdf)
found NBR to have the greatest signal to noise ratio with regard to forest disturbance
(signal) throughout the US.

```js
function calcNBR(img) {
  return img.normalizedDifference(['B5', 'B7']).rename('NBR');
}
```

In your own application, you may choose to use a different spectral index. Here
is the place to alter or add an additional index.

#### Combine functions

Define a wrapper function that can be mapped over all images in a collection to
apply all of the above functions.

```js
function prepImg(img) {
  var orig = img;
  img = oliBands(img);
  img = fmask(img);
  img = etm2oli(img);
  img = calcNBR(img);
  return ee.Image(img.copyProperties(orig, orig.propertyNames()));
}
```

In your application, you might decide to include or exclude functions. Alter this
wrapper as needed.

### Time Series Example

The `prepImg` wrapper function above can be mapped over Landsat surface reflectance
collections to create cross-sensor analysis ready data that can be used to visualize
the spectral chronology of a pixel or region of pixels. In this example you will
create a 35+ year time series and display the spectral history for a single pixel.
This particular pixel relates the recent history of a mature pacific northwest
conifer forest patch (Figure 1) that experiences some perturbation in the 1980's
and a high magnitude burn in the 2000s.

<figure id='fig1'>
  <img
		src="https://storage.googleapis.com/earthengine-community/tutorials/landsat-etm-to-oli-harmonization/area-of-interest.jpg"
		alt="Area of interest" style="width:100%">
  <figcaption>Figure 1. Location and site character for example area of interest.
		Mature pacific northwest conifer forest on the north slope of Mt Hood, OR, USA.</figcaption>
</figure>

#### Define an area of interest

The result of this application is a time series of Landsat observations for a pixel.
An `ee.Geometry.point` is used to define the pixel's location.

```js
var point = ee.Geometry.Point([-121.70938, 45.43185]);
```

In your application, you might select a different pixel. Simply, replace the above
longitude and latitude coordinates accordingly. Alternatively, you can summarize
the spectral history of group of pixels by using other `ee.Geometry` object definitions.
Please see the [Geometries](https://developers.google.com/earth-engine/geometries)
section of the Developer Guide for more information.

#### Retrieve Landsat sensor collections

Get Landsat USGS surface reflectance collections for
[OLI](https://developers.google.com/earth-engine/datasets/catalog/LANDSAT_LC08_C01_T1_SR),
[ETM+](https://developers.google.com/earth-engine/datasets/catalog/LANDSAT_LE07_C01_T1_SR), and
[TM](https://developers.google.com/earth-engine/datasets/catalog/LANDSAT_LT05_C01_T1_SR).<commit_msg>Image path change<commit_after>---
title: Landsat ETM+ to OLI Harmonization
description: Inter-sensor Landsat harmonization and time series visualization application.
author: jdbcode
tags: landsat, etm+, oli, tm, harmonization, time-series, join, reduce, visualization, fmask, cloud-mask, analysis-ready-data, ard
date_published: 2019-08-16
---
<!--
Copyright 2019 Google LLC

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->

[Open In Code Editor](https://code.earthengine.google.com/f06171a7bffc0a926d7e802b1f0580eb)

# Landsat ETM+ to OLI Harmonization

This tutorial concerns harmonizing Landsat ETM+ surface reflectance to Landsat OLI
surface reflectance and vice versa. It provides:

- A spectral translation function
- Functions to create analysis ready data
- A time series visualization example

The objective is to provide an end-to-end guide for visualizing a 35+ year regional
time series of Landsat data that can be immediately applied to your own region(s)
of interest.

Note that the characteristics of Landsat ETM+ and Landsat TM data are assumed equal
in this tutorial. As such, with regard to harmonization transformations, reference
to ETM+ is synonymous with TM.

## Context

[Landsat](https://landsat.gsfc.nasa.gov/about/) is a satellite imaging program
that has been collecting moderate resolution Earth imagery since 1972.
As the longest space-based Earth observation system, it provides a valuable
temporal record for identifying spatiotemporal trends in landscape change.

## Why harmonization?

[Roy et al. 2016](https://openprairie.sdstate.edu/cgi/viewcontent.cgi?referer=https://scholar.google.com/&httpsredir=1&article=1035&context=gsce_pubs)
demonstrate that there are small, but potentially significant differences between
the spectral characteristics of Landsat ETM+ and OLI, depending on application.
Reasons you might want to harmonize datasets include producing a long time series
that spans Landsat TM, ETM+, and OLI or you want to produce near-date intra-annual
composites to reduce the effects of missing observations from ETM+ SLC-off gaps
and masking clouds/shadows. Please see the linked manuscript above for more information.

## Instructions

### Functions

Define all the functions first.

#### Harmonization

Harmonization is achieved via linear translation of ETM+ spectral space to OLI
spectral space according to coefficients presented in
Roy et. al. (2016) Table 2 OLS regression. The following snippet defines a dictionary
containing slope (`slopes`) and intercept (`itcps`) images with respective slope
and intercept values per reflectance band.

```js
var coefficients = {
  itcps: ee.Image.constant([0.0003, 0.0088, 0.0061, 0.0412, 0.0254, 0.0172]).multiply(10000),
  slopes: ee.Image.constant([0.8474, 0.8483, 0.9047, 0.8462, 0.8937, 0.9071])
};
```

Band names for the same spectral response window vary between ETM+ (TM) and OLI
and should be unified. The following function selects only reflectance bands and
the `pixel_qa` bands from any given Landsat surface reflectance image and forces
all band names to be consistent with OLI.

```js
function oliBands(img) {
  return ee.Image(ee.Algorithms.If(
    ee.Algorithms.IsEqual(img.get('SATELLITE'), ee.String('LANDSAT_8')),
    img.select(['B2', 'B3', 'B4', 'B5', 'B6', 'B7', 'pixel_qa']),
    img.select(['B1', 'B2', 'B3', 'B4', 'B5', 'B7', 'pixel_qa'], ['B2', 'B3', 'B4', 'B5', 'B6', 'B7', 'pixel_qa'])
  ));
}
```

Finally, define the translation function, which applied the linear model, casts
type as Int16 for consistency with OLI, and reattaches the `pixel_qa` bands for
later use in cloud and shadow masking.

```js
function etm2oli(img) {
  return img.select(['B2', 'B3', 'B4', 'B5', 'B6', 'B7'])
    .multiply(coefficients.slopes)
    .add(coefficients.itcps)
    .round()
    .toShort()
    .addBands(img.select('pixel_qa'));
}
```

#### Cloud and shadow masking

Analysis ready data should have clouds and cloud shadows masked out. The following
function uses the CFmask ([Zhu et al, 2015](https://www.sciencedirect.com/science/article/abs/pii/S0034425714005069))
`pixel_qa` band included with each Landsat USGS surface reflectance image to set
pixels identified as cloud and cloud shadow to null.

```js
function fmask(img) {
  var cloudShadowBitMask = 1 << 3;
  var cloudsBitMask = 1 << 5;
  var qa = img.select('pixel_qa');
  var mask = qa.bitwiseAnd(cloudShadowBitMask).eq(0)
    .and(qa.bitwiseAnd(cloudsBitMask).eq(0));
  return img.updateMask(mask);
}
```

#### Spectral index calculation

Define a function to calculate normalized burn ratio (NBR). The following
application example looks at the spectral history of a forested pixel experiencing
a wildfire. NBR is used here because
[Cohen et al. (2018)](https://www.fs.fed.us/rm/pubs_journals/2018/rmrs_2018_cohen_w001.pdf)
found NBR to have the greatest signal to noise ratio with regard to forest disturbance
(signal) throughout the US.

```js
function calcNBR(img) {
  return img.normalizedDifference(['B5', 'B7']).rename('NBR');
}
```

In your own application, you may choose to use a different spectral index. Here
is the place to alter or add an additional index.

#### Combine functions

Define a wrapper function that can be mapped over all images in a collection to
apply all of the above functions.

```js
function prepImg(img) {
  var orig = img;
  img = oliBands(img);
  img = fmask(img);
  img = etm2oli(img);
  img = calcNBR(img);
  return ee.Image(img.copyProperties(orig, orig.propertyNames()));
}
```

In your application, you might decide to include or exclude functions. Alter this
wrapper as needed.

### Time Series Example

The `prepImg` wrapper function above can be mapped over Landsat surface reflectance
collections to create cross-sensor analysis ready data that can be used to visualize
the spectral chronology of a pixel or region of pixels. In this example you will
create a 35+ year time series and display the spectral history for a single pixel.
This particular pixel relates the recent history of a mature pacific northwest
conifer forest patch (Figure 1) that experiences some perturbation in the 1980's
and a high magnitude burn in the 2000s.

<figure id='fig1'>
  <img src="https://github.com/jdbcode/earthengine-community/blob/master/tutorials/landsat-etm-to-oli-harmonization/area-of-interest.jpg"
       alt="Area of interest" style="width:100%">
  <figcaption>Figure 1. Location and site character for example area of interest.
		Mature pacific northwest conifer forest on the north slope of Mt Hood, OR, USA.</figcaption>
</figure>

#### Define an area of interest

The result of this application is a time series of Landsat observations for a pixel.
An `ee.Geometry.point` is used to define the pixel's location.

```js
var point = ee.Geometry.Point([-121.70938, 45.43185]);
```

In your application, you might select a different pixel. Simply, replace the above
longitude and latitude coordinates accordingly. Alternatively, you can summarize
the spectral history of group of pixels by using other `ee.Geometry` object definitions.
Please see the [Geometries](https://developers.google.com/earth-engine/geometries)
section of the Developer Guide for more information.

#### Retrieve Landsat sensor collections

Get Landsat USGS surface reflectance collections for
[OLI](https://developers.google.com/earth-engine/datasets/catalog/LANDSAT_LC08_C01_T1_SR),
[ETM+](https://developers.google.com/earth-engine/datasets/catalog/LANDSAT_LE07_C01_T1_SR), and
[TM](https://developers.google.com/earth-engine/datasets/catalog/LANDSAT_LT05_C01_T1_SR).
{'fraction_non_alphanumeric': 0.09348353768594013, 'fraction_numerical': 0.029625941370510986, 'mean_word_length': 4.8599562363238515, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 15, 'https://': 24, 'lorem ipsum': 0, 'www.': 6, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3309181', 'n_tokens_mistral': 5142, 'n_tokens_neox': 4638, 'n_words': 1913}