<jupyter_start><jupyter_text>TensorBoard example We can integrate a server into Tensorboard so that we can see the development of our training sessions in real time as they are executing. In order to do that (see [Tensorboard documentation](https://jajimer.github.io/sinergym/compilation/html/pages/deep-reinforcement-learning.htmldrl-logger))We are going to use callbacks in order to do that.<jupyter_code>import sinergym
from sinergym.utils.callbacks import LoggerEvalCallback, LoggerCallback
from sinergym.utils.rewards import *
from sinergym.utils.wrappers import LoggerWrapper
from datetime import datetime
import gym
from stable_baselines3.common.logger import configure
from stable_baselines3 import DQN
from stable_baselines3.common.callbacks import CallbackList
from stable_baselines3.common.vec_env import DummyVecEnv<jupyter_output><empty_output><jupyter_text>You have to define a model like DRL notebook, this model will learn with the algorithm specified in the future.<jupyter_code>environment = "Eplus-demo-v1"
episodes = 4
experiment_date = datetime.today().strftime('%Y-%m-%d %H:%M')

# register run name
name = F"DQN-{environment}-episodes_{episodes}({experiment_date})"

env = gym.make(environment, reward=LinearReward)
env = LoggerWrapper(env)<jupyter_output>[2022-05-23 16:41:49,416] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2022-05-23 16:41:49,418] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2022-05-23 16:41:49,419] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...<jupyter_text>Now, we define tensorboard output path in the model definition.<jupyter_code>tensorboard_path='./tensorboard_log'
model = DQN('MlpPolicy', env, verbose=1, tensorboard_log=tensorboard_path)<jupyter_output>Using cpu device
Wrapping the env with a `Monitor` wrapper
Wrapping the env in a DummyVecEnv.<jupyter_text>Continue with the configuration as DRL notebook.<jupyter_code>n_timesteps_episode = env.simulator._eplus_one_epi_len / \
                      env.simulator._eplus_run_stepsize
timesteps = episodes * n_timesteps_episode
env_vec = DummyVecEnv([lambda: env])<jupyter_output><empty_output><jupyter_text>We define the `LoggerCallback` in order to specify how to store the real time training data.<jupyter_code>callbacks = []

# Set up Evaluation and saving best model
log_callback = LoggerCallback(True)
callbacks.append(log_callback)
# lets change default dir for TensorboardFormatLogger only
tb_path = tensorboard_path + '/' + name
new_logger = configure(tb_path, ["tensorboard"])
model.set_logger(new_logger)

callback = CallbackList(callbacks)<jupyter_output>2022-05-23 16:41:50.179466: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2022-05-23 16:41:50.179484: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.<jupyter_text>We have to include this callback in learning process.<jupyter_code>model.learn(
    total_timesteps=timesteps,
    callback=callback,
    log_interval=1)
model.save(env.simulator._env_working_dir_parent + '/' + name)
env.close()<jupyter_output>[2022-05-23 16:41:51,876] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-05-23 16:41:51,888] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res4/Eplus-env-sub_run1
[2022-05-23 16:42:11,278] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus episode completed successfully. 
[2022-05-23 16:42:11,279] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-05-23 16:42:11,291] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res4/Eplus-env-sub_run2
[2022-05-23 16:42:42,227] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus episode completed successfully. 
[2022-05-23 16:42:42,228] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-05-23 16:42:42,238] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /work[...]
{'fraction_non_alphanumeric': 0.0915858335213174, 'fraction_numerical': 0.0591021881344462, 'mean_word_length': 6.605488850771869, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '349975', 'license': 'MIT', 'n_tokens_mistral': 1656, 'n_tokens_neox': 1475, 'n_words': 357, 'path': 'examples/TensorBoard_example.ipynb', 'repo_name': 'jajimer/sinergym', 'size': '4433'}