<jupyter_start><jupyter_text>PyBPS Tutorial  1. InitializationThe first thing to do is obviously to import the `pybps` package.At the same time, we also import other useful packages.<jupyter_code>import pybps

import os
import sys
import re
import sqlite3
import pandas as pd
from pandas.io import sql
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>Once the `pybps` is imported into `python`, the first thing to do is to create an instance of the `BPSProject` class to get all the information required to run a particular simulation project from the project directory and hold it in a series of instance variables. This way, the project information will be easily retrieved when the different functions that manage simulation pre/post-processing and running are called.  The simplest and quickest way to get ready is to instanciate the `BPSProject` class with the `path` and `batch` arguments defined. `path` is the path (relative or absolute) to the directory holding the simulation files for a particular project. `batch` is a flag which sets whether the simulation project corresponds to a single run (`batch = False`, which is the default value) or to a batch run (`batch = True`).  In the present tutorial, we will use the very simple `Begin` example `TRNSYS` project that can be found in the `Examples` directory of any `TRNSYS` installation. We just made some modifications to the output (using Type46) and added parameters in an external `parameters.txt` file for the batch run. This example project can be found in the `Examples/TRNSYS` folder found in the `PyBPS` package. Note that as of today, `PyBPS` has only been tested with `TRNSYS` simulation projects, altought its functionnalities could easily be used with any other text-file-based simulation tool.<jupyter_code>bps = pybps.BPSProject('Examples/TRNSYS/Begin', batch=True)<jupyter_output><empty_output><jupyter_text>Another way to create an instance of the `BPSProject` class is to call it without any arguments and then use the `path` and `batch` methods to set both variables. In the case of the `batch` method, calling it sets `batch` to `True` (since by default it is set to `False`, which corresponds to a single simulation run).<jupyter_code>bps = pybps.BPSProject()
bps.path('Examples/TRNSYS/Begin')
bps.batch()<jupyter_output><empty_output><jupyter_text>Once we have got our `bps` object created, we can check the simulation project info obtained from the project folder and stored in the object's attributes. Behind the scenes, the `BPSProject` class uses two hidden methods to detect the simulation tool to be used (based on file extensions found in given directory) and to get the info required to run single or batch runs. The basic info needed to run any tool is contained in the `config.ini` file is the base folder of the `pybps` package.  If the project is of the "single run" type, the following instance variables hold the basic info needed to actually run the simulation:<jupyter_code># Path to the folder containing simulation files
bps.path
# Simulation tool name
bps.sim_tool
# Simulation input file path
bps.simfile_path
# Basic config info needed to call the proper commands to run the simulation tool and identify the basic simulation files.
# This info is contained in the "config.ini" file.
bps.config
# Particular configuration parameters can be acceded like in any python dictionnary
bps.config['executable']<jupyter_output><empty_output><jupyter_text>If the simulation project happens to be a batch run, an additional set of instance variables is created to hold information about the template and parameter files, as well as the list of jobs to be run. In `pybps`, template files are simulation files containing parameters to be replaced prior to running simulation. Parameters are identified as strings surrounded by `%` signs, like `%PAR%` for example. The user has to create the template files (replacing acordingly the simulation parameters with parameters search strings) prior to calling the `pybps` package and place a parameter file in csv format in the project folder. Template and parameter files should contain a specific search string in their filename to be recognized as such by `pybps`. By default, users should include `_TMP` in template filenames and `_PAR` in parameter filenames. These are just the default settings and can be modified in the `config.ini` file.If the simulation project was identified by the user as corresponding to a batch run (`batch = True`) and `pybps` can't find any template or parameter file, it will give an error message and exit.<jupyter_code># Unique ID for the batch to be run. Allows for succesive batch runs with different sets of parameters to be run within a same directory without the risk to overwrite cases.
# Also helps for storing info in sql databases
bps.series_id
# List of paths to the template files found in the project directory
bps.tmpfiles_pathlist
# Path to parameter file
bps.paramfile_path
# List of jobs to be run
# This is actually a list of dicts containing all the parameters for the current batch run
bps.job_list
# Number of simulation jobs to be run
bps.njob<jupyter_output><empty_output><jupyter_text>2. Pre-process Simulation Data<jupyter_code>job = pybps.BPSJob('Examples/TRNSYS/Begin_PARAM/SIM00001')
job.path<jupyter_output><empty_output><jupyter_text>3. Run Simulation Jobs<jupyter_code>bpsbatch.job[1].run()
bpsbatch.run()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.05490483162518302, 'fraction_numerical': 0.0021961932650073207, 'mean_word_length': 5.310623556581986, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '573913', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1443, 'n_tokens_neox': 1385, 'n_words': 774, 'path': 'pybps_tutorial/pybps_tutorial.ipynb', 'repo_name': 'dtavan/PyBPS', 'size': '5464'}