<jupyter_start><jupyter_text>Example of Using the Utility Bill data in a Notebook<jupyter_code>import pickle
import pandas as pd
import numpy as np
import benchmark         # the main benchmarking script file
import bench_util        # some utility functions related to benchmarking

# Just in case you want to plot something:
import matplotlib.pyplot as plt
 
# Show Plots in the Notebook
%matplotlib inline

# Increase the size of plots and their fonts
plt.rcParams['figure.figsize']= (10, 8)   # set Chart Size
plt.rcParams['font.size'] = 14            # set Font size in Chart

# Use a plotting style'style' the plot using 'bmh' style
plt.style.use('bmh')<jupyter_output><empty_output><jupyter_text>Using Data from the Last Run of the ScriptThis saves about 4 minutes of time because reading the raw data and pre-processing it is avoided.<jupyter_code># Read the raw Utility Bill data into a DataFrame
df_raw = pd.read_pickle('df_raw.pkl')

# Read the pre-processed data
dfp = pd.read_pickle('df_processed.pkl')

# Read in the bench_util.Util object, which provides some other data that may 
# be useful.
util_obj = pickle.load(open('util_obj.pkl', 'rb'))<jupyter_output><empty_output><jupyter_text>Take a look at what you got<jupyter_code>df_raw.head(3)
dfp.head(3)
# Get some other building info about the Big Dipper building
# A dictionary is returned.
util_obj.building_info('DIPMP1')<jupyter_output><empty_output><jupyter_text>Instead, Rerun the Preprocessing routine to get the DataFrames and Util object<jupyter_code># This calls the preprocessing function in the main benchmark script file.
# Values in the 'settings.py' file will determine which files are read.
# NOTE:  This will take 4 minutes to run, and the Jupyter Notebook does not 
# always show the messages that printed along the way, so it will look like
# your computer is frozen. Be patient.

# Uncomment the line below to run
# df_raw, dfp, util_obj = benchmark.preprocess_data()<jupyter_output><empty_output><jupyter_text>Look at Some Data to Illustrate the Duplicate kW Demand Charge Problem<jupyter_code># You can't use the Pandas "query" method for columns that have spaces in
# their names.  So use the more general, but cumbersome syntax to filter 
# the "Site ID" column.
# NOTE: a quick rename of the columns to eliminate spaces is another approach:
#  df_raw.columns = [c.replace(' ', '_') for c in df_raw.columns]
df_dip = df_raw[df_raw['Site ID']=='DIPMP1']

df_dip.query("From < '2017-06-01' and From > '2017-01-01' and Units=='kW'")
# Look at the processed data to compare Actual Demand vs. Demand Charge
df_dip = dfp.query("site_id=='DIPMP1' and units=='kW'")
df_pivot = pd.pivot_table(df_dip, index=['cal_year', 'cal_mo'], values='usage', columns='item_desc', aggfunc=np.sum)

# Do the following to show all rows of the DataFrame
with pd.option_context('display.max_rows', None):
    print(df_pivot)<jupyter_output>item_desc        Actual demand  Demand Charge   KW Charge  On peak demand
cal_year cal_mo                                                          
2006     6                 NaN            NaN  117.000000             NaN
         7                 NaN            NaN  234.000000             NaN
         8                 NaN            NaN  241.548387             NaN
         9                 NaN            NaN  287.701613             NaN
         10                NaN            NaN  320.078125             NaN
         11                NaN            NaN  289.821875             NaN
         12                NaN            NaN  285.521875             NaN
2007     1                 NaN            NaN  295.628125             NaN
         2                 NaN            NaN  271.307143             NaN
         3                 NaN            NaN  278.976190             NaN
         4                 NaN            NaN  266.223118             NaN
         5                 NaN        [...]
{'fraction_non_alphanumeric': 0.07082587573510611, 'fraction_numerical': 0.0383533623114293, 'mean_word_length': 2.2901597981497055, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 22, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '189962', 'license': 'MIT', 'n_tokens_mistral': 1271, 'n_tokens_neox': 1109, 'n_words': 417, 'path': 'using_benchmark_data.ipynb', 'repo_name': 'alanmitchell/fnsb-benchmark', 'size': '3911'}