<jupyter_start><jupyter_text>How to use custom preprocessing functions with `intake_esm`installed manually with `pip install git+https://github.com/NCAR/intake-esm.git` to get the latest version.<jupyter_code>import intake
import intake_esm
intake_esm.__version__<jupyter_output><empty_output><jupyter_text>A common annoyance with CMIP6 data is that not all instutions followed a uniform naming convention, resulting in problems when a user tries to loop over several models for comparison of certain fields.`cmip6_preprocessing` provides a renaming function which we can pass directly to `intake_esm` to get consistently named datasets.First consider a basic model selection (temperature data from historical run for 3 models on the native grid):<jupyter_code>query = dict(experiment_id='historical',# table_id='Oyr', 
             variable_id='thetao', grid_label='gn', member_id='r1i1p1f1',
             source_id=['MCM-UA-1-0', 'BCC-CSM2-MR', 'GFDL-ESM4'])
cat = col.search(**query)

dsets_pp = cat.to_dataset_dict()
for k, ds in dsets_pp.items():
    print(k)
    print(list(ds.dims))<jupyter_output>--> The keys in the returned dictionary of datasets are constructed as follows:
	'activity_id.institution_id.source_id.experiment_id.table_id.grid_label'

--> There will be 3 group(s)
CMIP.BCC.BCC-CSM2-MR.historical.Omon.gn
['bnds', 'lat', 'lev', 'lon', 'member_id', 'time']
CMIP.NOAA-GFDL.GFDL-ESM4.historical.Omon.gn
['bnds', 'lev', 'member_id', 'time', 'vertex', 'x', 'y']
CMIP.UA.MCM-UA-1-0.historical.Omon.gn
['bnds', 'latitude', 'lev', 'longitude', 'member_id', 'time']<jupyter_text>**Note the inhomogenous naming of variables**. In order to compare models we ideally want every dataset to have the same dimension names. We can achieve this by passing `rename_cmip6` as `preprocess` argument.<jupyter_code># Import custom renaming function
from cmip6_preprocessing.preprocessing import rename_cmip6

# pass to intake_esm
dsets_pp_renamed = cat.to_dataset_dict(preprocess=rename_cmip6)
for k, ds in dsets_pp_renamed.items():
    print(k)
    print(list(ds.dims))<jupyter_output>--> The keys in the returned dictionary of datasets are constructed as follows:
	'activity_id.institution_id.source_id.experiment_id.table_id.grid_label'

--> There will be 3 group(s)
CMIP.BCC.BCC-CSM2-MR.historical.Omon.gn
['bnds', 'lev', 'member_id', 'time', 'x', 'y']
CMIP.NOAA-GFDL.GFDL-ESM4.historical.Omon.gn
['bnds', 'lev', 'member_id', 'time', 'vertex', 'x', 'y']
CMIP.UA.MCM-UA-1-0.historical.Omon.gn
['bnds', 'lev', 'member_id', 'time', 'x', 'y']
{'fraction_non_alphanumeric': 0.1324685534591195, 'fraction_numerical': 0.009827044025157232, 'mean_word_length': 5.64490861618799, 'pattern_counts': {'":': 0, '<': 11, '<?xml version=': 0, '>': 15, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '518554', 'license': 'MIT', 'n_tokens_mistral': 928, 'n_tokens_neox': 891, 'n_words': 237, 'path': 'notebooks/raw_notebooks/tutorial_intake_esm_preprocessing.ipynb', 'repo_name': 'pedmondlerner/cmip6hack-ocean-bgc', 'size': '2544'}