<jupyter_start><jupyter_text>Computation ---  Learning Objectives - Do basic arithmetic with DataArrays and Datasets- Perform aggregation (reduction) along one or multiple dimensions of a DataArray or Dataset- Compute climatology and anomaly using xarray's "split-apply-combine" approach via `.groupby()`- Perform weighted reductions along one or multiple dimensions of a DataArray or Dataset  Prerequisites| Concepts | Importance | Notes || --- | --- | --- || [Understanding of xarray core data structures](./01-xarray-fundamentals.ipynb) | Necessary | || [Familiarity with xarray indexing and subsetting](./02-indexing-and-subsetting.ipynb) | Necessary | || [Familiarity with NumPy](https://numpy.org/doc/stable/reference/arrays.indexing.html) | Helpful | |- **Time to learn**: *20-30 minutes* ---  Imports<jupyter_code>import matplotlib.pyplot as plt
import numpy as np
import xarray as xr<jupyter_output><empty_output><jupyter_text>Let's open the monthly sea surface temperature data from the CESM2 model:<jupyter_code>ds = xr.open_dataset(
    "./data/tos_Omon_CESM2_historical_r11i1p1f1_gr_200001-201412.nc", engine="netcdf4"
)
ds<jupyter_output><empty_output><jupyter_text>Arithmetic OperationsArithmetic operations with a single DataArray automatically vectorize (like numpy) over all array values. Let's convert the air temperature from degree celsius to Kelvin:<jupyter_code>ds.tos + 273.15<jupyter_output><empty_output><jupyter_text>Lets's square all values in `tas`:<jupyter_code>ds.tos ** 2<jupyter_output><empty_output><jupyter_text>Aggregation Methods A very common step during data analysis is to summarize the data in question by computing aggregations like `sum()`, `mean()`, `median()`, `min()`, `max()` in which reduced data provide insight into the nature of large dataset. Let's explore some of these aggregation methods:<jupyter_code># Compute mean
ds.tos.mean()<jupyter_output><empty_output><jupyter_text>Because we specified no `dim` argument the function was applied over all dimensions. It is possible to specify a dimension along which to compute an aggregation. For example, to calculate the mean in time for all locations specify the time dimension as the dimension along which the mean should be calculated:<jupyter_code>ds.tos.mean(dim='time').plot(size=7, robust=True);
# compute temporal min
ds.tos.min(dim=['time'])
# compute spatial sum
ds.tos.sum(dim=['lat', 'lon'])
# compute temporal median
ds.tos.median(dim='time')<jupyter_output><empty_output><jupyter_text>The following table summarizes some other built-in Xarray aggregations:| Aggregation              | Description                     ||--------------------------|---------------------------------|| ``count()``              | Total number of items           || ``mean()``, ``median()`` | Mean and median                 || ``min()``, ``max()``     | Minimum and maximum             || ``std()``, ``var()``     | Standard deviation and variance || ``prod()``               | Compute product of elements            || ``sum()``                | Compute sum of elements                || ``argmin()``, ``argmax()``| Find index of minimum and maximum value |  GroupBy: Split, Apply, CombineSimple aggregations can give useful summary of our dataset, but often we would prefer to aggregate conditionally on some coordinate labels or groups. Xarray provides the so-called `groupby` operation which enables the **split / apply / combine** workflow on xarray DataArrays and Datasets. The split-apply-combine operation is illustrated in this figureThis makes clear what the `groupby` accomplishes:- The split step involves breaking up and grouping an xarray Dataset or DataArray depending on the value of the specified group key.- The apply step involves computing some function, usually an aggregate, transformation, or filtering, within the individual groups.- The combine step merges the results of these operations into an output xarray Dataset or DataArray.We are going to use `groupby` to remove the seasonal cycle ("climatology") from our dataset:<jupyter_code>ds.tos.sel(lon=310, lat=50, method='nearest').plot();<jupyter_output><empty_output><jupyter_text>SplitLet's group data by month i.e. all Januaries in one group, all Februaries in one group, etc...<jupyter_code>ds.tos.groupby(ds.time.dt.month)<jupyter_output><empty_output><jupyter_text>In the above example, we are using the `.dt` [`DatetimeAccessor`](https://xarray.pydata.org/en/stable/generated/xarray.core.accessor_dt.DatetimeAccessor.html) to extract specific components of dates/times in our time coordinate dimension<jupyter_code>ds.time.dt.year
ds.time.dt.month<jupyter_output><empty_output><jupyter_text>Xarray also offers a more concise syntax when the variable you’re grouping on is already present in the dataset. This is identical to `ds.tos.groupby(ds.time.dt.month)`:<jupyter_code>ds.tos.groupby('time.month')<jupyter_output><empty_output><jupyter_text>Apply & Combine Now that we have groups defined, it’s time to “apply” a calculation to the group. These calculations can either be:- aggregation: reduces the size of the group- transformation: preserves the group’s full sizeAt then end of the apply step, xarray will automatically combine the aggregated / transformed groups back into a single object.  Compute climatology Let's calculate the climatology at every point in the dataset:<jupyter_code>tos_clim = ds.tos.groupby('time.month').mean()
tos_clim
# Plot climatology at a specific point
tos_clim.sel(lon=310, lat=50, method='nearest').plot();
# Plot zonal mean climatology
tos_clim.mean(dim='lon').transpose().plot.contourf(levels=12, robust=True, cmap='turbo');
# Difference between January and December climatologies
(tos_clim.sel(month=1) - tos_clim.sel(month=12)).plot(size=6, robust=True);<jupyter_output><empty_output><jupyter_text>Compute anomalyNow let's combine the previous steps to compute climatology and use xarray's `groupby` arithmetic to remove this climatology from our original data<jupyter_code>gb = ds.tos.groupby('time.month')
tos_anom = gb - gb.mean(dim='time')
tos_anom
tos_anom.sel(lon=310, lat=50, method='nearest').plot();<jupyter_output><empty_output><jupyter_text>Let's look at the mean global anomaly in time by computing mean. We need to specify both `lat` and `lon` dimensions in the `dim` argument to mean:<jupyter_code>unweighted_mean_global_anom = tos_anom.mean(dim=['lat', 'lon'])
unweighted_mean_global_anom.plot();<jupyter_output><empty_output><jupyter_text>An operation which combines grid cells of different size is not scientifically valid unless each cell is weighted by the size of the grid cell. xarray has a convenient [`.weighted`](https://xarray.pydata.org/en/stable/user-guide/computation.htmlweighted-array-reductions) method to accomplish this Let's first load the cell area data. This dataset contains the weights for the grid cells<jupyter_code>areacello = xr.open_dataset("data/areacello_Ofx_CESM2_historical_r11i1p1f1_gr.nc").areacello
areacello<jupyter_output><empty_output><jupyter_text>As before, let's calculate area weighted mean global anomaly:<jupyter_code>weighted_mean_global_anom = tos_anom.weighted(areacello).mean(dim=['lat', 'lon'])<jupyter_output><empty_output><jupyter_text>Let's plot both unweighted and weighted means:<jupyter_code>unweighted_mean_global_anom.plot(size=7)
weighted_mean_global_anom.plot()
plt.legend(['unweighted', 'weighted']);<jupyter_output><empty_output><jupyter_text>---  Other high level computation functionality- `resample`: [Groupby-like functionality specifialized for time dimensions. Can be used for temporal upsampling and downsampling](https://xarray.pydata.org/en/stable/time-series.htmlresampling-and-grouped-operations)- `rolling`: [Useful for computing aggregations on moving windows of your dataset e.g. computing moving averages](https://xarray.pydata.org/en/stable/computation.htmlrolling-window-operations)- `coarsen`: [Generic functionality for downsampling data](https://xarray.pydata.org/en/stable/computation.htmlcoarsen-large-arrays)<jupyter_code># resample to annual frequency
r = ds.tos.resample(time='AS')
r
r.mean()
# Compute a 5-month moving average
m_avg = ds.tos.rolling(time=5, center=True).mean()
m_avg
lat = 50
lon = 310

m_avg.isel(lat=lat, lon=lon).plot(size=6)
ds.tos.isel(lat=lat, lon=lon).plot()
plt.legend(['5-month moving average', 'monthly data']);<jupyter_output><empty_output><jupyter_text>---<jupyter_code>%load_ext watermark
%watermark --time --python --updated --iversion<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11503697617091208, 'fraction_numerical': 0.008451696208475173, 'mean_word_length': 5.538756715272449, 'pattern_counts': {'":': 0, '<': 73, '<?xml version=': 0, '>': 73, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '216265', 'license': 'CC0-1.0', 'n_tokens_mistral': 2645, 'n_tokens_neox': 2537, 'n_words': 778, 'path': 'notebooks/04-computation.ipynb', 'repo_name': 'mgrover1/xarray-tutorial', 'size': '8519'}