<jupyter_start><jupyter_text>Accessing a NetCDF4/HDF5 File - S3 Direct Access  SummaryIn this notebook, we will access monthly sea surface height from ECCO V4r4 (10.5067/ECG5D-SSH44). The data are provided as a time series of monthly netCDFs on a 0.5-degree latitude/longitude grid.We will access a single netCDF file from inside the AWS cloud (us-west-2 region, specifically) and load it into Python as an `xarray` `dataset`. This approach leverages S3 native protocols for efficient access to the data. Requirements 1. AWS instance running in us-west-2NASA Earthdata Cloud data in S3 can be directly accessed via temporary credentials; this access is limited to requests made within the US West (Oregon) (code: us-west-2) AWS region. 2. Earthdata LoginAn Earthdata Login account is required to access data, as well as discover restricted data, from the NASA Earthdata system. Thus, to access NASA data, you need Earthdata Login. Please visit https://urs.earthdata.nasa.gov to register and manage your Earthdata Login account. This account is free to create and only takes a moment to set up. 3. netrc FileYou will need a netrc file containing your NASA Earthdata Login credentials in order to execute the notebooks. A netrc file can be created manually within text editor and saved to your home directory. For additional information see: [Authentication for NASA Earthdata](https://nasa-openscapes.github.io/2021-Cloud-Hackathon/tutorials/04_NASA_Earthdata_Authentication.htmlauthentication-via-netrc-file). Learning Objectives- how to retrieve temporary S3 credentials for in-region direct S3 bucket access- how to perform in-region direct access of ECCO_L4_SSH_05DEG_MONTHLY_V4R4 data in S3- how to plot the data ---  Import Packages<jupyter_code>%matplotlib inline
import matplotlib.pyplot as plt
import os
import requests
import s3fs
from osgeo import gdal
import xarray as xr
import hvplot.xarray
import holoviews as hv<jupyter_output><empty_output><jupyter_text>Get Temporary AWS CredentialsDirect S3 access is achieved by passing NASA supplied temporary credentials to AWS so we can interact with S3 objects from applicable Earthdata Cloud buckets. For now, each NASA DAAC has different AWS credentials endpoints. Below are some of the credential endpoints to various DAACs:<jupyter_code>s3_cred_endpoint = {
    'podaac':'https://archive.podaac.earthdata.nasa.gov/s3credentials',
    'gesdisc': 'https://data.gesdisc.earthdata.nasa.gov/s3credentials',
    'lpdaac':'https://data.lpdaac.earthdatacloud.nasa.gov/s3credentials',
    'ornldaac': 'https://data.ornldaac.earthdata.nasa.gov/s3credentials',
    'ghrcdaac': 'https://data.ghrc.earthdata.nasa.gov/s3credentials'
}<jupyter_output><empty_output><jupyter_text>Create a function to make a request to an endpoint for temporary credentials. Remember, each DAAC has their own endpoint and credentials are not usable for cloud data from other DAACs.<jupyter_code>def get_temp_creds(provider):
    return requests.get(s3_cred_endpoint[provider]).json()
temp_creds_req = get_temp_creds('podaac')
#temp_creds_req<jupyter_output><empty_output><jupyter_text>Set up an `s3fs` session for Direct Access`s3fs` sessions are used for authenticated access to s3 bucket and allows for typical file-system style operations. Below we create session by passing in the temporary credentials we recieved from our temporary credentials endpoint.<jupyter_code>fs_s3 = s3fs.S3FileSystem(anon=False, 
                          key=temp_creds_req['accessKeyId'], 
                          secret=temp_creds_req['secretAccessKey'], 
                          token=temp_creds_req['sessionToken'])<jupyter_output><empty_output><jupyter_text>In this example we're interested in the ECCO data collection from NASA's PO.DAAC in Earthdata Cloud. Below we specify the s3 URL to the data asset in Earthdata Cloud. This URL can be found via [Earthdata Search](../tutorials/01_Earthdata_Search.md) or programmatically  through the [CMR](https://nasa-openscapes.github.io/2021-Cloud-Hackathon/tutorials/01_Data_Discovery_CMR.html) and [CMR-STAC](https://nasa-openscapes.github.io/2021-Cloud-Hackathon/tutorials/02_Data_Discovery_CMR-STAC_API.html) APIs.<jupyter_code>s3_url = 's3://podaac-ops-cumulus-protected/ECCO_L4_SSH_05DEG_MONTHLY_V4R4/SEA_SURFACE_HEIGHT_mon_mean_2015-01_ECCO_V4r4_latlon_0p50deg.nc'<jupyter_output><empty_output><jupyter_text>Direct In-region AccessOpen with the netCDF file using the s3fs package, then load the cloud asset into an `xarray` `dataset`.<jupyter_code>s3_file_obj = fs_s3.open(s3_url, mode='rb')
ssh_ds = xr.open_dataset(s3_file_obj, engine='h5netcdf')
ssh_ds<jupyter_output><empty_output><jupyter_text>Get the `SSH` variable as an `xarray` `dataarray`<jupyter_code>ssh_da = ssh_ds.SSH
ssh_da<jupyter_output><empty_output><jupyter_text>Plot the `SSH` `dataarray` for time **2015-01-16T12:00:00** using `hvplot`.<jupyter_code>ssh_da.hvplot.image(x='longitude', y='latitude', cmap='Spectral_r', aspect='equal').opts(clim=(ssh_da.attrs['valid_min'][0],ssh_da.attrs['valid_max'][0]))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08395743003547497, 'fraction_numerical': 0.02148206543161214, 'mean_word_length': 5.62532637075718, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 9, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '469335', 'license': 'Apache-2.0', 'n_tokens_mistral': 1683, 'n_tokens_neox': 1607, 'n_words': 490, 'path': 'how-tos/additional_resources/Earthdata_Cloud__Single_File__Direct_S3_Access_NetCDF4_Example.ipynb', 'repo_name': 'NASA-Openscapes/2022-SWOT-Ocean-Cloud-Workshop', 'size': '5074'}