<jupyter_start><jupyter_text>Exploring the THREDDS catalog with Unidata's Siphon[Siphon](http://siphon.readthedocs.io/en/latest/) is a Python module for accessing data hosted on a THREDDS data server.Siphon works by parsing the catalog XML and exposing it with higher level functions.In this notebook we will explore data available on the Central & Northern California Ocean Observing System (CeNCOOS) THREDDS. The cell below extracts the catalog information<jupyter_code>from siphon.catalog import TDSCatalog

catalog = TDSCatalog('https://thredds.cencoos.org/thredds/catalog.xml')


info = """
Catalog information
-------------------

Base THREDDS URL: {}
Catalog name: {}
Catalog URL: {}
Metadata: {}
""".format(catalog.base_tds_url,
           catalog.catalog_name,
           catalog.catalog_url,
           catalog.metadata)

print(info)<jupyter_output>Catalog information
-------------------

Base THREDDS URL: https://thredds.cencoos.org
Catalog name: CeNCOOS
Catalog URL: https://thredds.cencoos.org/thredds/catalog.xml
Metadata: {}<jupyter_text>Unfortunately this catalog has no metadata. So let's check what kind of services are available.<jupyter_code>for service in catalog.services:
    print(service.name)<jupyter_output>all
allandsos
wms<jupyter_text>And what datasets are there?<jupyter_code>print('\n'.join(catalog.datasets.keys()))<jupyter_output>California Coastal Regional Ocean Modeling System (ROMS) Nowcast
California Coastal Regional Ocean Modeling System (ROMS) Forecast
Monterey Bay (MB) Regional Ocean Modeling System (ROMS) Nowcast
Monterey Bay (MB) Regional Ocean Modeling System (ROMS) Forecast
Southern California Bight (SCB) Regional Ocean Modeling System (ROMS) Nowcast
UCSC California Current System Model
HAB Cellular Domoic Acid Forecast
HAB Cellular Domoic Acid Nowcast
HAB Particulate Domoic Acid Forecast
HAB Particulate Domoic Acid Nowcast
HAB Pseudo Nitzschia Forecast
HAB Pseudo Nitzschia Nowcast
Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS) Relative Humidity
Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS) Total Precipitation
Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS) Visibility
Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS) Wind 10 m
Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS) Air Temp 2 m
Coupled Ocean/Atmosphere Mesoscale Predictio[...]<jupyter_text>It looks like model runs as well as satellite and HFR data. One can also check the catalog refs for more information<jupyter_code>print('\n'.join(catalog.catalog_refs.keys()))
ref = catalog.catalog_refs['Global']

[value for value in dir(ref) if not value.startswith('__')]
info = """
Href: {}
Name: {}
Title: {}
""".format(
    ref.href,
    ref.name,
    ref.title)

print(info)<jupyter_output>Href: https://thredds.cencoos.org/thredds/global.xml
Name: 
Title: Global<jupyter_text>The `follow` method navigates to that catalog `ref` and returns a new `siphon.catalog.TDSCatalog` object for that part of the THREDDS catalog.<jupyter_code>cat = ref.follow()

print(type(cat))<jupyter_output><class 'siphon.catalog.TDSCatalog'><jupyter_text>That makes it easier to explore a small subset of the datasets available in the catalog.Here are the data from the *Global* subset.<jupyter_code>print('\n'.join(cat.datasets.keys()))<jupyter_output>NCEP Reanalysis Daily Averages Surface Flux
Global 1-km Sea Surface Temperature (G1SST)
NCEP Global Forecast System Model (GFS)
Aquarius V 3.0 Scatterometer Daily Aggregate
Aquarius V 3.0 Scatterometer Seven-Day Aggregate
Aquarius V 3.0 Scatterometer Monthly Aggregate
Aquarius V 3.0 Radiometer Daily Aggregate
Aquarius V 3.0 Radiometer Seven-Day Aggregate
Aquarius V 3.0 Radiometer Monthly Aggregate
Aquarius V 4.0 Scatterometer Daily Aggregate
Aquarius V 4.0 Scatterometer Seven-Day Aggregate
Aquarius V 4.0 Scatterometer Monthly Aggregate
Aquarius V 4.0 Radiometer Daily Aggregate
Aquarius V 4.0 Radiometer Seven-Day Aggregate
Aquarius V 4.0 Radiometer Monthly Aggregate<jupyter_text>Let's extract the `Global 1-km Sea Surface Temperature` dataset from the global `ref`.<jupyter_code>dataset = 'Global 1-km Sea Surface Temperature (G1SST)'

ds = cat.datasets[dataset]

ds.name, ds.url_path<jupyter_output><empty_output><jupyter_text>Siphon has a `ncss` (NetCDF subset service) access, here is a quote from the documentation:> This module contains code to support making data requests tothe NetCDF subset service (NCSS) on a THREDDS Data Server (TDS). This includesforming proper queries as well as parsing the returned data.Let's check if the catalog offers the `NetcdfSubset` in the `access_urls`.<jupyter_code>for name, ds in catalog.datasets.items():
    if ds.access_urls:
        print(name)<jupyter_output>California Coastal Regional Ocean Modeling System (ROMS) Nowcast
California Coastal Regional Ocean Modeling System (ROMS) Forecast
Monterey Bay (MB) Regional Ocean Modeling System (ROMS) Nowcast
Monterey Bay (MB) Regional Ocean Modeling System (ROMS) Forecast
Southern California Bight (SCB) Regional Ocean Modeling System (ROMS) Nowcast
UCSC California Current System Model
HAB Cellular Domoic Acid Forecast
HAB Cellular Domoic Acid Nowcast
HAB Particulate Domoic Acid Forecast
HAB Particulate Domoic Acid Nowcast
HAB Pseudo Nitzschia Forecast
HAB Pseudo Nitzschia Nowcast
Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS) Relative Humidity
Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS) Total Precipitation
Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS) Visibility
Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS) Wind 10 m
Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS) Air Temp 2 m
Coupled Ocean/Atmosphere Mesoscale Predictio[...]<jupyter_text>All `access_urls` returned empty.... Maybe that is just a metadata issue because there is `NetcdfSubset` access when navigating in the webpage.<jupyter_code>from IPython.display import HTML


iframe = '<iframe src="{src}" width="800" height="550" style="border:none;"></iframe>'.format

url = 'https://thredds.cencoos.org/thredds/catalog.html?dataset=G1_SST_US_WEST_COAST'
HTML(iframe(src=url))<jupyter_output><empty_output><jupyter_text>To finish the post let's check if there is any WMS service available and overlay the data in a slippy (interactive) map.<jupyter_code>services = [service for service in catalog.services if service.name == 'wms']

services<jupyter_output><empty_output><jupyter_text>Found only one, let's tease that out and check the URL.<jupyter_code>service = services[0]

url = service.base

url<jupyter_output><empty_output><jupyter_text>OWSLib helps to inspect the available layers before plotting. Here we will get the first layer that has G1_SST_US_WEST_COAST on it.Note, however, we are skipping the discovery step of the `wms` information and hard-coding it instead.That is to save time because parsing the URL [http://pdx.axiomalaska.com/ncWMS/wms](http://pdx.axiomalaska.com/ncWMS/wms) takes ~ 10 minutes. See [this](https://github.com/ioos/notebooks_demos/pull/171issuecomment-271705056) issue for more information.<jupyter_code>from owslib.wms import WebMapService


if False:
    web_map_services = WebMapService(url)
    layer = [key for key in web_map_services.contents.keys() if 'G1_SST_US_WEST_COAST' in key][0]
    wms = web_map_services.contents[layer]

    title = wms.title
    lon = (wms.boundingBox[0] + wms.boundingBox[2]) / 2.
    lat = (wms.boundingBox[1] + wms.boundingBox[3]) / 2.
    time = wms.defaulttimeposition
else:
    layer = 'G1_SST_US_WEST_COAST/analysed_sst'
    title = 'Sea Surface Temperature'
    lon, lat = -122.50, 39.50
    time = 'undefined'
import folium

m = folium.Map(location=[lat, lon], zoom_start=4)

folium.WmsTileLayer(
    name='{} at {}'.format(title, time),
    url=url,
    layers=layer,
    fmt='image/png',
    transparent=True,
).add_to(m)

folium.LayerControl().add_to(m)

m<jupyter_output><empty_output><jupyter_text>Last but not least a static image for the page thumbnail.<jupyter_code>from IPython.display import Image

Image(m._to_png())<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08811929997535124, 'fraction_numerical': 0.009489770766576288, 'mean_word_length': 5.492, 'pattern_counts': {'":': 0, '<': 49, '<?xml version=': 0, '>': 50, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 3}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '290569', 'license': 'MIT', 'n_tokens_mistral': 2737, 'n_tokens_neox': 2620, 'n_words': 826, 'path': 'notebooks/2017-01-18-siphon-explore-thredds.ipynb', 'repo_name': 'kellydesent/notebooks_demos', 'size': '8114'}