<jupyter_start><jupyter_text>Example 2 - Custom TemplateExample 1 showed how a pre-defined recipe could be used to build up a model - in this case around semi-lumped catchments generating runoff and constituent loads.This example looks at how a new recipe might be built, starting with a template for a repeatable model unit.Where the SemiLumpedCatchment allowed some configuration by the user (in terms of constituents, HRUs, CGUs), here we will lock down the configuration as you might do when building a bespoke solution.Here, we will look at a model that estimates hillslope fine sediment. The template might be applied at different spatial scales, ranging from grid cell to subcatchment.<jupyter_code>from openwater.discovery import discover
from openwater import OWTemplate, OWLink
import openwater.nodes as n
import openwater.template as templating

import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
discover()<jupyter_output><empty_output><jupyter_text>Constructing the templateWe will apply the template to every grid cell in a map.The template will represent rainfall runoff and hillslope fine sediment generation. Then there will be a transport process for moving the sediment downstream.We won't go all the way through parameterisation. Just the template and then applying the template to the mapWe build everything up in an `OWTemplate` object<jupyter_code>template = OWTemplate()<jupyter_output><empty_output><jupyter_text>Now we can add various nodes. A node simply represents a point where a model algorithm will run. (so, even routing processes are made up of nodes).The main thing is that we specify a model (eg `n.GR4J`) but we can also add adhoc tags. When we apply the template to the map, we'll add additional tags (like row, column, or in other cases, catchment)<jupyter_code>sediment_node = template.add_node(n.USLEFineSedimentGeneration,process='Hillslope')
runoff_node = template.add_node(n.GR4J,process='Runoff')
scale_node = template.add_node(n.DepthToRate,process='RunoffScaling')<jupyter_output><empty_output><jupyter_text>We also need to specify the connections between models **within** the template. These connections are **links** - simply representing the movement of data between two nodes. There is no transformation on links. (Although its worth noting that when two links feed into the one variable on a single node, their values are **summed**)<jupyter_code>template.add_link(OWLink(runoff_node,'runoff',scale_node,'input'))

template.add_link(OWLink(scale_node,'outflow',sediment_node,'quickflow'))<jupyter_output><empty_output><jupyter_text>Now we add one more node and link, to represent the routing process. In this template we really just want this to help us link up multiple instances of the template between catchments:<jupyter_code>transport_node = template.add_node(n.LumpedConstituentRouting,process='Transport')
template.add_link(OWLink(sediment_node,'totalLoad',transport_node,'lateralLoad'))<jupyter_output><empty_output><jupyter_text>Spatial StructureWe will construct the model by applying the template to every grid cell in (fabricated) map.In this case, the map will be square. We need a flow direction grid to show where things accumulate. We will say that any point North of the NW-SE diagonal will flow south (5). Any point to the west of the diagonal will flow east (3) and any point on the diagonal will flow Sout East (4).<jupyter_code>def dummy_pointer_grid(sz):
    arr = np.ones(sz**2).reshape(sz,sz)
    for row,col in list(zip(*np.where(arr))):
        if row < col:
            arr[row,col] = 5
        elif col < row:
            arr[row,col] = 3
        else:
            arr[row,col] = 4
    return arr
GRID_SIZE=10
arr = dummy_pointer_grid(GRID_SIZE)
plt.imshow(arr)
plt.colorbar()<jupyter_output><empty_output><jupyter_text>Build the model graphWe now combine the pointer map, with the model template to come up with a single model graph, `g`.This will represent the template in every grid cell. So, we have 100 grid cells resulting in 100 GR4J nodes, etc.We build it up incrementally, **and add relevant tags as we go**. In this case, we add row and col tags<jupyter_code>g = None

for row,col in list(zip(*np.where(arr))):
    g = templating.template_to_graph(g,template,row=row,col=col)<jupyter_output><empty_output><jupyter_text>At this point, out graph contains 100 copies of the template, with row and col tags.Now we need to link them up. Here we look back at the pointer map to see what direction a grid cell flows (and we just include E, SE, and S options!)From the direction we can work out the destination grid cell and hence the destination node. We then add a link between the `LumpedConstituentRouting` nodes in each grid cell<jupyter_code>for row,col in list(zip(*np.where(arr))):
    ptr = arr[row,col]
    if ptr==3:
        dest_row = row
        dest_col = col+1
    elif ptr==5:
        dest_row = row+1
        dest_col = col
    else:
        dest_row = row+1
        dest_col = col+1

    if dest_row >= GRID_SIZE:
        continue
    src_node = templating.match_nodes(g,_model='LumpedConstituentRouting',row=row,col=col)[0]
    dest_node = templating.match_nodes(g,_model='LumpedConstituentRouting',row=dest_row,col=dest_col)[0]
    
    g.add_edge(src_node,dest_node,src=['outflowLoad'],dest=['inflowLoad'])<jupyter_output><empty_output><jupyter_text>Now we have the graph, we can convert it to an OpenWater model graph, which includes scheduling the models to run in different batches<jupyter_code>model = templating.ModelGraph(g)
stage_lengths = [len(s) for s in model.order]
plt.plot(stage_lengths,label='sl')
print(stage_lengths)<jupyter_output>[19, 36, 51, 64, 56, 48, 40, 32, 24, 16, 9, 4, 1]
{'fraction_non_alphanumeric': 0.07798566182899108, 'fraction_numerical': 0.009442210176604302, 'mean_word_length': 5.157158234660926, 'pattern_counts': {'":': 0, '<': 38, '<?xml version=': 0, '>': 37, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '617519', 'license': 'MIT', 'n_tokens_mistral': 1725, 'n_tokens_neox': 1661, 'n_words': 662, 'path': 'doc/Example-1-2-CustomTemplate.ipynb', 'repo_name': 'flowmatters/openwater', 'size': '5719'}