<jupyter_start><jupyter_text>Supercompressible (7d)<NAME> (<EMAIL>)\September 22, 2020This notebook creates the **design of experiments** and the metadata required to run the **numerical simulations**.**Note**: the parametric `Abaqus` scripts were created using `f3dasm` implementations. You can also create your own functions and use them within this framework!**TODO**: * add notebooks are not the best tool to run this (use script), but this intends to show how to use the tools. Let's import the required libraries.<jupyter_code># imports

# standard library
import os
import pickle
from collections import OrderedDict

# third-party
import numpy as np
from SALib.sample import sobol_sequence
import pandas as pd
from matplotlib import pyplot as plt

# local library
from f3dasm.design_of_experiments.transform_inputs import transform_inputs_supercompressible
from f3dasm.run.utils import create_main_file
from f3dasm.run.utils import get_sims_info
from f3dasm.run.stats import analyze_times
from f3dasm.abaqus.run.utils import run_sims
from f3dasm.post_processing.utils import post_process_sims
from f3dasm.post_processing.supercompressible import get_results
from f3dasm.post_processing.utils import concatenate_data<jupyter_output><empty_output><jupyter_text>Now define the directory name and the number of points.<jupyter_code>example_name = 'example_4_1'

n_pts = 10  # number of points
import shutil
try:
    shutil.rmtree(example_name)
except:
    pass<jupyter_output><empty_output><jupyter_text>A quick verification of the chosen name will be performed to avoid overriding existing folders.<jupyter_code>if os.path.exists(example_name):
    raise Exception('Name already exists')<jupyter_output><empty_output><jupyter_text>Introduction**TODO*** What is f3dasm?  Design of experiments Now, we need to define the problem.Let's define the **variables of the problem**, both the variables that are defined by the design of experiments scheme and the fixed variables (i.e. the variables that are common for all simulations).**Note**: the use of an ```OrderedDict``` is not mandatory, but forces variable order in the ```Pandas``` frame.<jupyter_code>doe_variables = OrderedDict({'ratio_area': [1.17e-5, 4.1e-3],
                             'ratio_Ixx': [1.128e-11, 1.4e-6],
                             'ratio_Iyy': [1.128e-11, 1.4e-6],
                             'ratio_J': [1.353e-11, 7.77e-6],
                             'ratio_pitch': [.25, 1.5],
                             'ratio_top_diameter': [0., .8],
                             'ratio_shear_modulus': [.035, .45]})

fixed_variables = {'n_longerons': 3,
                   'bottom_diameter': 100.,
                   'young_modulus': 3500.}<jupyter_output><empty_output><jupyter_text>The supercompressible problem is *special*, in the way there's an additional variable that is not fixed nor controlled by the design of experiments scheme: **imperfections**.<jupyter_code>seed = 1
deg2rad = np.pi / 180
m = 4. * deg2rad  # mean
s = 1.2 * deg2rad  # std
sigma = np.sqrt(np.log(s**2 / m**2 + 1))
mu = np.log((m**2) / np.sqrt(s**2 + m**2))
imperfection_dist = {'mean': mu, 'sigma': sigma}
imperfections = np.random.lognormal(size=n_pts, **imperfection_dist)<jupyter_output><empty_output><jupyter_text>Another important step is the definition of the information related with the numerical simulations.<jupyter_code>sim_info = {'abstract_model': ['f3dasm.abaqus.models.supercompressible_fnc.lin_buckle',
                               'f3dasm.abaqus.models.supercompressible_fnc.riks'],
            'sim_info': OrderedDict({'SUPERCOMPRESSIBLE_LIN_BUCKLE':
                                     {'job_info': {'name': 'Simul_supercompressible_lin_buckle',
                                                   'description': ''}},
                                     'SUPERCOMPRESSIBLE_RIKS':
                                     {'job_info': {'name': 'Simul_supercompressible_riks',
                                                   'description': ''}}}),
            'post_processing_fnc': ['f3dasm.abaqus.models.supercompressible_fnc.post_process_lin_buckle',
                                   'f3dasm.abaqus.models.supercompressible_fnc.post_process_riks'],
            'transform_inputs': transform_inputs_supercompressible}<jupyter_output><empty_output><jupyter_text>The simulation definition is quite straightforward.First, we need to define the `abstract_model`. That is, the class that is initialized given the variables values and contains all the methods required to successfully create, run and post-process the numerical model.Second, we need to define the specific parameters of each simulation. In this example, there's a first simulation that performs a linear buckling analysis and another that performs a riks analysis. The second simulation depends on the results of the first. The use of an `OrderedDict` is of great importance to guarantee the right order of the simulations. `job_info` is the fundamental key and has to be always defined.Finally, we define also a transformation of inputs. This is simply a mapping from the variables used in the design of experiments to the variables required to create the model. Now, using a proper design of experiments scheme, we can create the design of experiments. Here, we simply use Sobol sequence implementation from ```SALib``` library.<jupyter_code>dim = len(doe_variables)
points_sobol = sobol_sequence.sample(n_pts, dim)
for i, lims in enumerate(doe_variables.values()):
    points_sobol[:, i] = points_sobol[:, i] * (lims[1] - lims[0]) + lims[0]<jupyter_output><empty_output><jupyter_text>Finally, we store the information in a dictionary and save it. Note the design of experiments is stored in a ```Pandas.DataFrame```. This is a convenient way for the subsequent application of Machine Learning.<jupyter_code>doe_variables_ls = list(doe_variables.keys())
points = pd.DataFrame(points_sobol, columns=doe_variables_ls)
additional_variables = {'imperfection': imperfections}
additional_info = {'imperfection_dist': imperfection_dist,
                   'seed': seed}
create_main_file(example_name, doe_variables, points, sim_info,
                 fixed_variables=fixed_variables,
                 additional_variables=additional_variables,
                 additional_info=additional_info)<jupyter_output><empty_output><jupyter_text>Run simulations First, let's open the created file and see what it contains.<jupyter_code>with open(os.path.join(example_name, 'DoE.pkl'), 'rb') as file:
    data = pickle.load(file)
    
print(data.keys())<jupyter_output>dict_keys(['doe_variables', 'points', 'sim_info', 'run_info', 'version', 'fixed_variables', 'additional_variables', 'additional_info'])<jupyter_text>For the running step, the field `run_info` is of great importance, because it gives information about the simulations that are still missing or were already run (both successfully or with errors).<jupyter_code>print(data['run_info'])<jupyter_output>{'missing_sims': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], 'running_sims': [], 'error_sims': [], 'successful_sims': []}<jupyter_text>During the running we can see simulations status using `get_sims_info` (it also works before and after).<jupyter_code>get_sims_info(example_name)<jupyter_output>Missing simulations (total): 10/10 (100.0%)<jupyter_text>We run the simulations through `run_sims`. We can control the simulations we want to run or choose to run the first `n_sims`.We can also use parallel computating to run several simulations simultaneously (only allowed if each simulation uses only one cpu). In that case, a `jupyter notebook` may not be the most appropriate tool because all the code has necessarily to be within the condition `if __name__ == '__main__'`. e.g.:```pythonfrom f3dasm.abaqus.run.utils import run_simsif __name__ == '__main__':     (...)        run_sims(example_name)```<jupyter_code>run_sims(example_name, points=[1], abaqus_path='abaqus',
         keep_odb=True, dump_py_objs=False, gui=False)<jupyter_output><empty_output><jupyter_text>Post-processing After running the simulations, we collect data and store the quantities of interest in the `pandas.Dataframe` that contains the design of experiments. We have to provide a function that transforms the data collected during the simulations (contained in the key `'post-processing'`).<jupyter_code>post_process_sims(pp_fnc=get_results,
                  output_variables=['coilable', 'sigma_crit', 'energy'],
                  example_name=example_name)<jupyter_output><empty_output><jupyter_text>with open(os.path.join(example_name, 'DoE.pkl'), 'rb') as file:    data = pickle.load(file)    print(data['points']) Finally, it may be useful to merge all the collected information in a unique file. That's achieved using `concatenate_data`. We could also have done in an inverted way: first concatenate data and then post-process using the created dictionary.<jupyter_code>concatenate_data(example_name, delete=False)<jupyter_output><empty_output><jupyter_text>With `f3dasm` we can also perform a statistic analysis of the running times (that may be useful to identify weak points in the numerical simulation).<jupyter_code>analyze_times(example_name, raw_data='raw_data.pkl');
with open(os.path.join(example_name, 'DoE.pkl'), 'rb') as file:
    data = pickle.load(file)

print(data['points'])<jupyter_output>ratio_area     ratio_Ixx     ratio_Iyy       ratio_J  ratio_pitch  \
0    0.000012  1.128000e-11  1.128000e-11  1.353000e-11     0.250000   
1    0.002056  7.000056e-07  7.000056e-07  3.885007e-06     0.875000   
2    0.003078  3.500085e-07  3.500085e-07  1.942510e-06     1.187500   
3    0.001034  1.050003e-06  1.050003e-06  5.827503e-06     0.562500   
4    0.001545  5.250071e-07  8.750042e-07  6.798752e-06     0.718750   
5    0.003589  1.225001e-06  1.750099e-07  2.913758e-06     1.343750   
6    0.002567  1.750099e-07  1.225001e-06  4.856255e-06     1.031250   
7    0.000523  8.750042e-07  5.250071e-07  9.712618e-07     0.406250   
8    0.000778  4.375078e-07  1.312501e-06  3.399383e-06     0.953125   
9    0.002822  1.137502e-06  6.125063e-07  7.284376e-06     0.328125   

   ratio_top_diameter  ratio_shear_modulus coilable sigma_crit energy  
0                0.00             0.035000     None       None   None  
1                0.40             0.242500        1    72.8026 [...]
{'fraction_non_alphanumeric': 0.09505446833124458, 'fraction_numerical': 0.054757543622867055, 'mean_word_length': 3.7939001848428835, 'pattern_counts': {'":': 0, '<': 59, '<?xml version=': 0, '>': 59, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '73877', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 3524, 'n_tokens_neox': 3178, 'n_words': 946, 'path': 'examples/supercompressible_fnc_7d/example_supercompressible_7d.ipynb', 'repo_name': 'bessagroup/F3DASM', 'size': '10391'}