<jupyter_start><jupyter_text>2020-10-18 Exploring Why So many Big Ms.The PTL DQN Agent gets so many Big Ms versus the Random Agent that I think something might be wrong with the logging (duplicate logging due to experience replay maybe?<jupyter_code># Common data analysis
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns;sns.set(color_codes=True)
import scipy.stats as st

# Python
import datetime
from pathlib import Path
import os

# For display
from IPython.display import display, Markdown, Latex


# To stop dataframes from compressing
pd.set_option('display.max_colwidth', None)
pd.set_option('display.max_row', 5000)
pd.set_option('display.max_columns', None)

# To change resolution to match retina
%config InlineBackend.figure_format = 'retina'

#Set base dir
# Idea gotten from https://stackoverflow.com/questions/39125532/file-does-not-exist-in-jupyter-notebook
project_dir = Path(globals()['_dh'][0]+"/..").resolve()
os.chdir(project_dir)
print("Running notebook from: " + os.path.abspath(""))


from IPython.core.interactiveshell import InteractiveShell
#InteractiveShell.ast_node_interactivity = "last_expr" # Default jupyter behavior
InteractiveShell.ast_node_interactivity = "all" # All expressions are shown.

# OPTIONAL: Load the "autoreload" extension so that code can change
%load_ext autoreload

# OPTIONAL: always reload modules so that as you change code in src, it gets loaded
%autoreload 2

plt.rc("figure", dpi=150)

#TO DO: convert this into a template.<jupyter_output>Running notebook from: /Users/aleph/Documents/jota/tesis/ts_mcfrl
The autoreload extension is already loaded. To reload it, use:
  %reload_ext autoreload<jupyter_text>Loading Data<jupyter_code>dqn_movement_detail_report = pd.read_csv("python/data/results/dqn_few_warehouses_v4__demandgen_biased/movement_detail_report.csv")
dqn_valid_dcs = pd.read_csv("python/data/results/dqn_few_warehouses_v4__demandgen_biased/valid_dcs.csv")

custdqn_movement_detail_report = pd.read_csv("python/data/results/dqn_onehot_few_warehouses_v4/movement_detail_report.csv")

random_movement_detail_report = pd.read_csv("python/data/results/gr_random_few_warehouses/movement_detail_report.csv")
random_valid_dcs = pd.read_csv("python/data/results/gr_random_few_warehouses_debug/valid_dcs.csv")<jupyter_output><empty_output><jupyter_text>Comparing Big Ms in dqn<jupyter_code>dqn_movement_detail_report.groupby(['episode','is_big_m']).size().reset_index().head()<jupyter_output><empty_output><jupyter_text>No duplicates to be seen on the ep 0, so it may seem that it's the dqn that's to blame for these Big Ms. Are we looking at the same data?<jupyter_code>dqn_movement_detail_report.query('episode==0').query('is_big_m').sort_values(['source_time','destination_time','destination_name']).head(5)<jupyter_output><empty_output><jupyter_text>Comparing the valid movement dataset on each<jupyter_code>display(dqn_valid_dcs.groupby('customer').agg(list).reset_index().sort_values('customer').head(10))
display(random_valid_dcs.groupby('customer').agg(list).reset_index().sort_values('customer').head(10))<jupyter_output><empty_output><jupyter_text>So why does the random make better decisions in a 3 warehouse scenario environment? Something's weird: the DQN has the 5 expected orders per step, random only has one.<jupyter_code>dqn_movement_detail_report.query('episode==0').query("movement_type=='Delivery'").sort_values(['source_time','destination_time','destination_name','source_name']).head(25)
custdqn_movement_detail_report.query('episode==0').query("movement_type=='Delivery'").sort_values(['source_time','destination_time','destination_name','source_name']).head(25)
random_movement_detail_report.query('episode==0').query("movement_type=='Delivery'").sort_values(['source_time','destination_time','destination_name','source_name']).head(25)# this one doesnt match<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11434405308831036, 'fraction_numerical': 0.01174068402246044, 'mean_word_length': 8.605392156862745, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '380978', 'license': 'MIT', 'n_tokens_mistral': 1363, 'n_tokens_neox': 1282, 'n_words': 263, 'path': 'notebooks/2020-10-18 Unexpected Too Many BigM DQN.ipynb', 'repo_name': 'jotaporras/ts_mcfrl', 'size': '3918'}