<jupyter_start><jupyter_text>TMIP-EMAT Running Core Model Experiments on Multiple ComputersExperiments designed for sensitivity testing and meta-model development are run using the core model (as opposed to a meta-model). These experiments are completely independent and thus can be run in parallel if multiple computers are available. In this situation, an initial process is needed to create the TMIP-EMAT database, populate it with the scope, and design the experiments. At this point, jupyter notebooks on separate computers can connect to the same database to access experiment designs and to import metrics from the core model run.The notebook cells below demonstrate the initial configuration that is only run **once**, that involves: 1. Creating a TMIP-EMAT database2. Storing the scope in the database3. Designing a set of experimentsThe following section demonstrates the execution steps that are run **on each modeling computer**, these involve:1. **Connecting** to the TMIP-EMAT database2. **Reading** the scope from the database3. Running a subset of the experiments<jupyter_code>import emat
emat.__version__
import pandas, numpy, os
from emat.util.loggers import log_to_stderr
log = log_to_stderr(level=20)
scope = emat.Scope('GBNRTC_scope.yaml') # scope file is local to the Master<jupyter_output><empty_output><jupyter_text>Create SQLite DB file and record scopeThe database file needs to be stored in a location accessible by all modeling systems<jupyter_code>db = emat.SQLiteDB("emat_db/gbnrtc_multi.db",initialize=True)
scope = emat.Scope("gbnrtc_scope.yaml")<jupyter_output><empty_output><jupyter_text>Load Model<jupyter_code>from emat.model import GBNRTCModel
g = GBNRTCModel(
    configuration='gbnrtc_model_config.yaml',
    scope=scope,
    db=db,
)
g<jupyter_output><empty_output><jupyter_text>Design experimentsOnly done by Master notebook<jupyter_code>design = g.design_experiments(scope=scope,
                              n_samples_per_factor=10, 
                              sampler='lhs',
                              design_name='test_lhs')
design.head()<jupyter_output><empty_output><jupyter_text>Run subset of experiments on each computerResults will be stored in the database associated with the model***The following cells are to be run in an independent notebook***<jupyter_code>import emat
emat.__version__

import pandas, numpy, os
from emat.util.loggers import log_to_stderr
log = log_to_stderr(level=20)
#scope = emat.Scope('GBNRTC_scope.yaml') # Scope will be read from the database<jupyter_output><empty_output><jupyter_text>Connect to existing SQLite DB file and read scopeFile created by Master notebook (step 1.1 must be complete by the Master notebook)<jupyter_code>db = emat.SQLiteDB("emat_db/gbnrtc.db",initialize=False)
scope = db.read_scope('GBNRTC')<jupyter_output><empty_output><jupyter_text>Load Model<jupyter_code>from emat.model import GBNRTCModel
g = GBNRTCModel(
    configuration='gbnrtc_model_config.yaml',
    scope=scope,
    db=db,
)
g<jupyter_output><empty_output><jupyter_text>Run subset of experimentsRun set of experiments (1-35)Results will be stored in the database associated with the model<jupyter_code>design_subset = g.read_experiments(design_name='gbnrtc_lhs').iloc[0:35]
lhs_results = g.run_experiments(design_subset)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07426337943475646, 'fraction_numerical': 0.006013229104028864, 'mean_word_length': 5.680722891566265, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '440599', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 992, 'n_tokens_neox': 963, 'n_words': 319, 'path': 'docs/source/emat.examples/GBNRTC/gbnrtc_multisystem.ipynb', 'repo_name': 'tlumip/tmip-emat', 'size': '3326'}