<jupyter_start><jupyter_text>ADM1F_SRT: Input/output sensitivityHere we explore the relationships between inputs and outputs. In the `ADM1F: Execution time` example we showed how to run the models with the perturbed input values from  `influent.dat` and `param.dat` files. Assuming that you run the `ADM1F: Execution time` example and produced the `outputs_influent.csv` and `outputs_params.csv` files, we use these outputs here to study the relationship between influents and outputs, and params and outputs. If not just uncomment lines 5 and 18 and re-run the simulations. Authors: <NAME> and <NAME><jupyter_code>import adm1f_utils as adm1fu
import os
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline<jupyter_output><empty_output><jupyter_text>1. Influent/Output sensitivity<jupyter_code># navigate to simulations folder
os.chdir('../simulations')
#Set the path to the ADM1F executable
ADM1F_EXE = 'path-to-your-ADM1F-folder/build/adm1f'

# Set the value of percentage and sample size for lhs
percent = 0.1 # NOTE: for params percent should be <= 0.05 
sample_size = 100
variable = 'influent'    # influent/params/ic
method = 'uniform'    #'uniform' or 'lhs'
index=adm1fu.create_a_sample_matrix(variable,method,percent,sample_size)
print ()
print ('Number of elements participated in the sampling:',len(index))
#exe_time=adm1fu.adm1f_output_sampling(ADM1F_EXE,variable,index)
[output_name,output_unit]=adm1fu.get_output_names()
alloutputs = pd.read_csv('outputs_influent.csv', sep=',', header=None)
alloutputs.columns = output_name
alloutputs
[influent_name,influent_index]=adm1fu.get_influent_names()
# since we did not use all the columns in the influent.dat (see create_a_sample_matrix) 
# we use index to select used headers for used values
header=[]
for i in index:
    header.append(influent_name[i])

influent_inputs = pd.read_csv('var_influent.csv', sep=',', header=None)
influent_inputs.columns = header
influent_inputs.head()
# merge influent and output datasets
inout=pd.concat([influent_inputs,alloutputs], axis=1)
inout.head()<jupyter_output><empty_output><jupyter_text>The correlation heat map matrix below shows that four influents have the highest impact on the results: `X_ch_biom_in`, `X_pr_biom_in`, `X_li_biom_in`, and `Q`.<jupyter_code>corr=inout.corr()
plt.figure(figsize=(21,5))
sns.heatmap(corr.iloc[0:11,11:-1], cmap=sns.diverging_palette(220, 10, as_cmap=True))
plt.title('Correlation Matrix [influent/results]',fontsize=16);
plt.ylabel('Influent',fontsize=16)
plt.xlabel('Outputs',fontsize=16)<jupyter_output><empty_output><jupyter_text>Let's merge `ph` values from the results with the influents and explore the correlations.<jupyter_code>influent_ph=pd.concat([influent_inputs,alloutputs[' pH ']], axis=1)
plt.figure(figsize=(10,4))
influent_ph.corr().iloc[-1].plot(linewidth=2)
plt.title('Correlation plot (ph)')
plt.xlabel('Influents')
plt.ylabel('Correlation')
plt.ylim([-1,1])
plt.scatter( influent_ph['X_pr_biom_in'],influent_ph[' pH '])
plt.xlabel('X_pr_biom_in',fontsize=18)
plt.ylabel('pH',fontsize=18);
from pandas.plotting import scatter_matrix
scatter_matrix(influent_ph, alpha=0.6,figsize=(10,10));<jupyter_output><empty_output><jupyter_text>2. Params/Output sensitivity<jupyter_code># Set the value of percentage and sample size for lhs
percent = 0.1 # NOTE: for params percent should be <= 0.05 
sample_size = 100
variable = 'params'    # influent/params/ic
method = 'uniform'    #'uniform' or 'lhs'
index=adm1fu.create_a_sample_matrix(variable,method,percent,sample_size)
print ()
print ('Number of elements participated in the sampling:',len(index))
#exe_time=adm1fu.adm1f_output_sampling(ADM1F_EXE,variable,index)
[output_name,output_unit]=adm1fu.get_output_names()
alloutputs = pd.read_csv('outputs_params.csv', sep=',', header=None)
alloutputs.columns = output_name
[param_name,param_index]=adm1fu.get_param_names()
# since we did not use all the columns in the influent.dat (see create_a_sample_matrix) 
# we use index to select used headers for used values
header=[]
for i in index:
    header.append(param_name[i])

param_inputs = pd.read_csv('var_params.csv', sep=',', header=None)
param_inputs.columns = header
param_inputs.head()
# merge influent and output datasets
inout=pd.concat([param_inputs,alloutputs], axis=1)
inout.head()
corr=inout.corr()
plt.figure(figsize=(25,15))
sns.heatmap(corr.iloc[0:92,92:-1], cmap=sns.diverging_palette(220, 10, as_cmap=True))
plt.title('Correlation Matrix [params/results]',fontsize=16);
plt.ylabel('Parameters',fontsize=16)
plt.xlabel('Outputs',fontsize=16)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12194594594594595, 'fraction_numerical': 0.02227027027027027, 'mean_word_length': 7.5036764705882355, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 23, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '367158', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1691, 'n_tokens_neox': 1609, 'n_words': 381, 'path': 'notebooks/io_sensitivity_analysis.ipynb', 'repo_name': 'lanl/ADM1F_LANL', 'size': '4640'}