<jupyter_start><jupyter_text>Procedural flow defined by recipe<jupyter_code>!esmvaltool recipes get examples/recipe_python.yml
print(open('recipe_python.yml', 'r').read())
!esmvaltool run recipe_python.yml<jupyter_output>/home/stef/miniconda3/envs/esmvaltool/lib/python3.8/site-packages/iris/config.py:139: UserWarning: Ignoring config item 'Resources':'test_data_dir' (section:option) as '/home/h05/itwl/projects/git/iris-test-data/test_data' is not a valid directory path.
  warnings.warn(msg.format(section, option, c_path))
2021-02-16 16:07:29,367 UTC [15495] INFO    
______________________________________________________________________
          _____ ____  __  ____     __    _ _____           _
         | ____/ ___||  \/  \ \   / /_ _| |_   _|__   ___ | |
         |  _| \___ \| |\/| |\ \ / / _` | | | |/ _ \ / _ \| |
         | |___ ___) | |  | | \ V / (_| | | | | (_) | (_) | |
         |_____|____/|_|  |_|  \_/ \__,_|_| |_|\___/ \___/|_|
______________________________________________________________________

ESMValTool - Earth System Model Evaluation Tool.

http://www.esmvaltool.org

CORE DEVELOPMENT TEAM AND CONTACTS:
  <NAME> (Co-PI; DLR, Germany - <EMAIL>)
  <NAME> (C[...]<jupyter_text>Output is difficult to navigate<jupyter_code>!tree /home/stef/esmvaltool_output/`ls -t /home/stef/esmvaltool_output/ | head -n1`<jupyter_output>[01;34m/home/stef/esmvaltool_output/recipe_python_20210216_160729[00m
├── index.html
├── [01;34mplots[00m
│   ├── [01;34mmap[00m
│   │   └── [01;34mscript1[00m
│   │       └── [01;34mpng[00m
│   │           ├── [01;35mCMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000.png[00m
│   │           ├── CMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000_citation.bibtex
│   │           ├── CMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000_data_citation_info.txt
│   │           ├── [01;35mCMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000_provenance.svg[00m
│   │           ├── CMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000_provenance.xml
│   │           ├── [01;35mCMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_2000-2000.png[00m
│   │           ├── CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_2000-2000_citation.bibtex
│   │           ├── CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_2000-2000_data_citation_info.txt
│   │           ├── [01;35mCMIP6_BCC-ESM1_Amon_histo[...]<jupyter_text>Recipe metadataIn the ESMValTool project, authors are defined by tags, i.e. `lastname_firstname`.The tags point to entries in a dict in a large yaml config.Look-up and resolve author tags:<jupyter_code>from esmvalcore.experimental.recipe_metadata import Contributor

author = Contributor.from_tag('doe_john')
print(f'{author.name=}')
print(f'{author.institute=}')
print(f'{author.orcid=}')<jupyter_output>/home/stef/r/esmvalcore/esmvalcore/experimental/_warnings.py:18: UserWarning: 
  Thank you for trying out the new ESMValCore API.
  Note that this API is experimental and may be subject to change.
  More info: https://github.com/ESMValGroup/ESMValCore/issues/498
/home/stef/miniconda3/envs/esmvaltool/lib/python3.8/site-packages/iris/config.py:139: UserWarning: Ignoring config item 'Resources':'test_data_dir' (section:option) as '/home/h05/itwl/projects/git/iris-test-data/test_data' is not a valid directory path.
/home/stef/r/esmvalcore/esmvalcore/experimental/config/_config_validators.py:254: ESMValToolDeprecationWarning: `write_plots` will be removed in 2.4.0.
/home/stef/r/esmvalcore/esmvalcore/experimental/config/_config_validators.py:255: ESMValToolDeprecationWarning: `write_netcdf` will be removed in 2.4.0.<jupyter_text>Reference tags point to a bibtex file in the ESMValTool directory. They are loaded and rendered using `pybtex`.<jupyter_code>from esmvalcore.experimental.recipe_metadata import Reference
reference = Reference.from_tag('brunner2019')
print(reference)<jupyter_output><NAME>, <NAME>, <NAME>, and <NAME>. Quantifying uncertainty in european climate projections using combined performance-independence weighting. Environmental Research Letters, 14(12):124010, nov 2019. URL: https://doi.org/10.1088%2F1748-9326%2Fab492f, doi:10.1088/1748-9326/ab492f.<jupyter_text>Rendering output in a Jupyter Notebookhttps://ipython.readthedocs.io/en/stable/api/generated/IPython.display.htmlFrontends for:- HTML- JSON- PNG- JPEG- SVG- LaTeX<jupyter_code>class MyClass:
    pass

my_class = MyClass()
my_class
class MyClass:
    def __repr__(self):
        return 'My cool class 😎'

my_class = MyClass()
my_class<jupyter_output><empty_output><jupyter_text>IPython looks for `_repr_$_` methods<jupyter_code>class MyClass:
    def __repr__(self):
        return 'My cool class'
    
    def _repr_markdown_(self):
        return """# My cool markdown header"""

my_class = MyClass()
my_class<jupyter_output><empty_output><jupyter_text>Jinja2 as a templating enginehttps://jinja.palletsprojects.com/Jinja is a modern and designer-friendly templating language for Python, modelled after Django’s templates. It is fast, widely used and secure.<jupyter_code>from jinja2 import Template

template = Template('Hello {{ author.name }}!')
template.render(author=author)
template = Template("""
<h2>my list</h2>

<ul>
  {% for number in numbers %}
     <li>{{ number }}</li>
  {% endfor %}
</ul>
""")

rendered = template.render(numbers=('one', 'two', 'three', 'four'))
print(rendered)
from IPython.display import HTML
HTML(rendered)<jupyter_output><empty_output><jupyter_text>Combining Jinja2 with `_repr_html_`<jupyter_code>class Linkifier:
    template = Template('<a href="{{ url }}">Click here!</a>')
    
    def __init__(self, url):
        self.url = url
    
    def _repr_html_(self):
        return self.template.render(url=self.url)
        
my_class = Linkifier('https://www.esciencecenter.nl')
my_class<jupyter_output><empty_output><jupyter_text>Rendering authors and references in a notebook:<jupyter_code>author = Contributor.from_tag('doe_john')
author
reference = Reference.from_tag('brunner2019')
reference<jupyter_output><empty_output><jupyter_text>Rendering recipe metadataAll metadata in the recipe documentation can be resolved and rendered.<jupyter_code>from esmvalcore.experimental import get_recipe

recipe = get_recipe('examples/recipe_python.yml')
recipe<jupyter_output><empty_output><jupyter_text>Searching for recipesBecause all the tags are resolved, it is possible to search for recipes.<jupyter_code>from esmvalcore.experimental import get_all_recipes

recipes = get_all_recipes()
recipes<jupyter_output><empty_output><jupyter_text>`get_all_recipes` returns an instance of `RecipeList`, which has a `find` method.This makes it possible to search recipes by author institute:<jupyter_code>recipes_by_nlesc = recipes.find('NLeSC')
recipes_by_nlesc<jupyter_output><empty_output><jupyter_text>And refine that list by author name:<jupyter_code>recipes_by_stef = recipes_by_nlesc.find('Stef Smeets')
recipes_by_stef
recipe = recipes_by_stef[0]
recipe<jupyter_output><empty_output><jupyter_text>Running a recipe<jupyter_code>from esmvalcore.experimental import Recipe

recipe = Recipe('recipe_python.yml')
output = recipe.run()
output = recipe.get_output()
output<jupyter_output><empty_output><jupyter_text>Image output<jupyter_code>image_file = output['timeseries/script1'].image_files[0]
image_file<jupyter_output><empty_output><jupyter_text>Access provenance information:<jupyter_code>image_file.attributes<jupyter_output><empty_output><jupyter_text>Access citation info:<jupyter_code>print(image_file.citation_file.read_text())<jupyter_output>@article{righi20gmd,
	doi = {10.5194/gmd-13-1179-2020},
	url = {https://doi.org/10.5194/gmd-13-1179-2020},
	year = {2020},
	month = mar,
	publisher = {Copernicus {GmbH}},
	volume = {13},
	number = {3},
	pages = {1179--1199},
	author = {<NAME> and <NAME> and <NAME> and <NAME> and <NAME> and <NAME> and <NAME> and <NAME> and Bj"{o}<NAME>"{o}tz and <NAME> and <NAME> and <NAME> and <NAME> and <NAME> and <NAME> and <NAME> and <NAME> and <NAME> and <NAME>},
	title = {Earth System Model Evaluation Tool (ESMValTool) v2.0 -- technical overview},
	journal = {Geoscientific Model Development}
}

@misc{acknow_project,
    title = "Please acknowledge the project(s)."
}

@misc{https://doi.org/10.22033/ESGF/CMIP6.2949,
	url = {https://doi.org/10.22033/ESGF/CMIP6.2949},
	title = {BCC BCC-ESM1 model output prepared for CMIP6 CMIP historical},
	publisher = {Earth System Gr[...]<jupyter_text>Access data citation info:<jupyter_code>print(image_file.data_citation_file.read_text())<jupyter_output>Follow the links below to find more information about CMIP6 data:
- https://cera-www.dkrz.de/WDCC/ui/cerasearch/cmip6?input=CMIP6.CMIP.BCC.BCC-ESM1.historical

Additional data citation information was found, for which no entry is available in the bibtex file:
- http://www.cccma.ec.gc.ca/models<jupyter_text>Data output<jupyter_code>data_file = output['timeseries/script1'].data_files[0]
data_file<jupyter_output><empty_output><jupyter_text>Load the data using `iris`<jupyter_code>cubelist = data_file.load_iris()
cubelist[0]<jupyter_output><empty_output><jupyter_text>Load the data using `xarray`<jupyter_code>arr = data_file.load_xarray()
arr<jupyter_output><empty_output><jupyter_text>Customizing an ESMValTool run<jupyter_code>from esmvalcore import CFG
CFG
CFG.start_session('my_session')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12589428723972237, 'fraction_numerical': 0.0407901761879338, 'mean_word_length': 5.445973847212663, 'pattern_counts': {'":': 0, '<': 117, '<?xml version=': 0, '>': 117, 'https://': 9, 'lorem ipsum': 0, 'www.': 4, 'xml': 1}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '22', 'dir': 'jupyter-structured-clean-dedup', 'id': '397205', 'license': 'Apache-2.0', 'n_tokens_mistral': 3803, 'n_tokens_neox': 3478, 'n_words': 627, 'path': 'presentation.ipynb', 'repo_name': 'stefsmeets/ESMValCore-Notebooks', 'size': '9556'}