<jupyter_start><jupyter_text>Opening the Intake Catalog<jupyter_code>import numpy as np
%matplotlib inline
import matplotlib.pyplot as plt
import intake<jupyter_output><empty_output><jupyter_text>We can load the catalog file with `intake.open_catalog()`. The catalog contains the information needed to access the data for each field in the dataset. You can get a list of the different fields in the dataset with `list()`.<jupyter_code># dset = "Darwin_v0.1_llc90"
#catalog = intake.open_catalog("catalog/"+dset + '.yml')
catalog = intake.open_catalog("Darwin_v0.1_llc90.yml")
catalog = catalog
fldnames = list(catalog)
print(fldnames)<jupyter_output>['ALK', 'CDOM', 'C_DIN', 'C_Fe', 'C_PO4', 'C_Si', 'Chl01', 'Chl02', 'Chl03', 'Chl04', 'Chl05', 'Chl06', 'Chl07', 'Chl08', 'Chl09', 'Chl10', 'Chl11', 'Chl12', 'Chl13', 'Chl14', 'Chl15', 'Chl16', 'Chl17', 'Chl18', 'Chl19', 'Chl20', 'Chl21', 'Chl22', 'Chl23', 'Chl24', 'Chl25', 'Chl26', 'Chl27', 'Chl28', 'Chl29', 'Chl30', 'Chl31', 'Chl32', 'Chl33', 'Chl34', 'Chl35', 'DIC', 'DOC', 'DOFe', 'DOP', 'DRHODR', 'Denit', 'ETAN', 'EVELMASS', 'EVELSTAR', 'EXFwspee', 'FeT', 'GGL90Kr', 'GGL90Lmx', 'GGL90TKE', 'MXLDEPTH', 'NH4', 'NO2', 'NO3', 'NVELMASS', 'NVELSTAR', 'Nfix', 'O2', 'PAR', 'PARF', 'PHIBOT', 'PIC', 'PO4', 'POC', 'POFe', 'PON', 'POP', 'POSi', 'PP', 'Rirr001', 'Rirr002', 'Rirr003', 'Rirr004', 'Rirr005', 'Rirr006', 'Rirr007', 'Rirr008', 'Rirr009', 'Rirr010', 'Rirr011', 'Rirr012', 'Rirr013', 'SALT', 'SFLUX', 'SIarea', 'SIatmFW', 'SIatmQnt', 'SIheff', 'SIhsnow', 'S_DIN', 'S_Fe', 'S_PO4', 'S_Si', 'SiO2', 'THETA', 'WVELMASS', 'WVELSTAR', 'a001', 'a002', 'a003', 'a004', 'a005', 'a006', 'a007', 'a0[...]<jupyter_text>Retrieving and Plotting Data We'll pick a field, c01 concentration. Each entry in the catalog can be accessed like a dictionary, with the field name as the key. Then we can get some basic information and metadata about this field with the `describe()` function.<jupyter_code>fldname = "c01" # c01 concentration
catalog_entry = catalog[fldname]

catalog_entry.describe()<jupyter_output><empty_output><jupyter_text>Once we load the catalog we can use the `read_chunked()` function on a catalog entry to read in a particular field. At this point we have an xarray `DataArray` object.Then you can create variables that point to the data on OpenDap. This does not download data at this point. Data is only downloaded when you specify a "slice".<jupyter_code>dataset
# Access the file
dataset = catalog_entry.read_chunked()

# Dimensions and Variable
time = dataset["tim"]
depth = dataset["dep_c"]
lat = dataset["lat_c"]
lon = dataset["lon_c"]
TRAC21 = dataset["c01"]

# See what this looks like
TRAC21
TRAC21.long_name<jupyter_output><empty_output><jupyter_text>We can look at the attributes of a variable and get the individual attributes of a variable. We can also find out how many elements a variable has and its shape.<jupyter_code># Attributes of variables
print("Attributes of c01: " + str(TRAC21.attrs))

# Get a particular attribute
print('\n' + "Units of depth: " + depth.attrs["units"] + '\n')
# or depth.units also works

# Inspect size and shape
print("c01 has " + str(TRAC21.size) + " elements and a shape of " + str(TRAC21.shape))<jupyter_output>Attributes of c01: {'long_name': 'c01 concentration', 'units': 'mmol C/m^3', '_ChunkSizes': [1, 9, 216, 432]}

Units of depth: m

c01 has 312478560000 elements and a shape of (2679, 50, 1080, 2160)<jupyter_text>When you request a slice of data it will be downloaded. Let's look at the surface of the 500th time step.<jupyter_code>t = 500; d = 0;
TRAC21_data = TRAC21[0,d,:,:].data

TRAC21_data<jupyter_output><empty_output><jupyter_text>Let's plot some things!<jupyter_code>lat_ticks = np.arange(90, -91, step=-30)
lon_ticks = np.arange(-180, 181, step=60)
title_txt = '\ndepth = ' + str(depth[d].data) + ' ' + depth.units  + '\ntime =  '+ str(time[t].data)

a = np.flipud(TRAC21_data.reshape(TRAC21_data.shape[0],TRAC21_data.shape[1]))
plt.imshow(a, cmap='hot', interpolation='nearest')
plt.colorbar()
plt.xticks(np.arange(0,TRAC21_data.shape[1],356),lon_ticks)
plt.yticks(np.arange(0,TRAC21_data.shape[0],179),lat_ticks)
plt.xlabel('Longitude')
plt.ylabel('Latitude')
plt.title(TRAC21.long_name +' (' + TRAC21.units + ')' + title_txt)
plt.show()<jupyter_output><empty_output><jupyter_text>Working with Data To show how to work with the data, we'll show how to get the total phytoplankton concentration for a particular time and depth. Note this section takes a few minutes to run.<jupyter_code>flds = ["TRAC"+str(n) for n in range(21,72)]
t = 500; d = 0;
import time as tm
start = tm.time()
dset = "darwin_v0.2_cs510"
f = "TRAC21" # c01 concentration
dataset = catalog[f].read_chunked()
thresh = .0000001

time = dataset["time"]
depth = dataset["dep_c"]

total_phyto = dataset[f][t,d,:,:].data
total_phyto[total_phyto == -9.9999998e+33] = np.nan
for f in flds[1:-1]:
    dataset = catalog[f].read_chunked()
    fld = dataset[f][t,d,:,:].data
    #fld[fld < thresh] = 0
    total_phyto = total_phyto + fld

end = tm.time()
print ("Time elapsed:", end - start)
lat_ticks = np.arange(90, -91, step=-30)
lon_ticks = np.arange(-180, 181, step=60)
title_txt = '\ndepth = ' + str(depth[d].data) + ' ' + depth.units  + '\ntime =  '+ str(time[t].data)

a = np.flipud(total_phyto.reshape(total_phyto.shape[0],total_phyto.shape[1]))
plt.imshow(a, cmap='hot', interpolation='nearest')
plt.colorbar()
plt.xticks(np.arange(0,total_phyto.shape[1],356),lon_ticks)
plt.yticks(np.arange(0,total_phyto.shape[0],179),lat_ticks)
plt.xlabel('Longitude')
plt.ylabel('Latitude')
plt.title('Total Phytoplankton (' + TRAC21.units + ')' + title_txt)
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.16906854130052723, 'fraction_numerical': 0.05922671353251318, 'mean_word_length': 6.025925925925926, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 31, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '38834', 'license': 'MIT', 'n_tokens_mistral': 2469, 'n_tokens_neox': 2228, 'n_words': 571, 'path': 'LAS_example_Python.ipynb', 'repo_name': 'christophernhill/lauren-cbiomes-las-notebooks', 'size': '5690'}