<jupyter_start><jupyter_text>Hakase's OMIP 1deg run TODO? update to Hakase's new data location https://arccss.slack.com/archives/C6PP0GU9Y/p1614814982063600 Hi all. Just to let you know that the two OMIP runs with ACCESS-OM2-BGC have now been transferred and stored at ik11:/g/data/ik11/outputs/access-om2/1deg_jra55_iaf_omip2_cycle[1-6]/g/data/ik11/outputs/access-om2/1deg_jra55_iaf_omip2spunup_cycle[1-40]Consequently, I deleted the data on scratch/v45.<jupyter_code>dbname = '1deg_jra55_iaf_omip2-fixed.db'
sess_1deg = cc.database.create_session(dbname)
#cc.database.build_index(['/scratch/v45/hh0162/access-om2/archive/1deg_jra55_iaf_omip2-fixed'], sess_1deg, update=True)
cc.database.build_index(glob.glob('/scratch/v45/aek156/access-om2/archive/1deg_jra55_iaf_omip2-fixed_cycle*'), sess_1deg, update=True, followsymlinks=True)
glob.glob('/scratch/v45/aek156/access-om2/archive/1deg_jra55_iaf_omip2-fixed_cycle*')
cc.querying.get_experiments(sess_1deg, all=True)<jupyter_output><empty_output><jupyter_text>01deg_jra55v140_iaf_cycle3<jupyter_code>dbname = '/g/data/v45/aek156/notebooks/github/aekiss/CC_sandbox/cyc3_database_analysis3-20p07.db'
cyc3_session = cc.database.create_session(dbname)
cyc3_dir_list = glob.glob('/scratch/x77/aek156/access-om2/archive/01deg_jra55v140_iaf_cycle3') + glob.glob('/scratch/v45/aek156/access-om2/archive/01deg_jra55v140_iaf_cycle3')
cc.database.build_index(cyc3_dir_list, cyc3_session, update=True)<jupyter_output><empty_output><jupyter_text>Parameter ensemble<jupyter_code>ens_session = cc.database.create_session('/home/156/aek156/payu/param_ensemble/param_ensemble.db') # for conda analysis3-20.10
ens_dir_list = glob.glob('/scratch/jk72/aek156/access-om2/archive/[01]*deg*')
cc.database.build_index(ens_dir_list, ens_session, update=True, followsymlinks=True)
ens_session = cc.database.create_session('/home/156/aek156/payu/param_ensemble/param_ensemble_update.db') # for conda analysis3-21.04
ens_dir_list = glob.glob('/scratch/jk72/aek156/access-om2/archive/[01]*deg*')
cc.database.build_index(ens_dir_list, ens_session, update=True, followsymlinks=True)<jupyter_output><empty_output><jupyter_text>DB for missing JRA55 variables - REDUNDANT?See https://github.com/COSIMA/cosima-cookbook/issues/207issuecomment-773066616<jupyter_code>dbname = 'jra_extras.db'
jra_extras_session = cc.database.create_session(dbname)

jra_extras_dir_list = ['/g/data/qv56/replicas/input4MIPs/CMIP6/OMIP/MRI/MRI-JRA55-do-1-4-0/atmos/3hrPt/vas/gr/v20190429']
cc.database.build_index(jra_extras_dir_list, jra_extras_session, update=True)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12924382716049382, 'fraction_numerical': 0.0787037037037037, 'mean_word_length': 8.066433566433567, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '239681', 'license': 'Apache-2.0', 'n_tokens_mistral': 1185, 'n_tokens_neox': 1081, 'n_words': 98, 'path': 'update_db.ipynb', 'repo_name': 'aekiss/ice_analysis', 'size': '2592'}