<jupyter_start><jupyter_text>Generate Profiles of MISO Sub-AreasMISO: [Midcontinent Independent System Operator](https://www.misoenergy.org/)The historical forecasted and actual load data rolled up at an hourly level and grouped by *Local Resource Zone* (LRZ)  for 2015 and 2016 have been downloaded [here](https://www.misoenergy.org/markets-and-operations/real-time--market-data/market-reports).<jupyter_code>import pandas as pd<jupyter_output><empty_output><jupyter_text>Read 2016 MISO load data.<jupyter_code>lrz_demand_2016 = pd.read_excel('20161231_dfal_hist.xls', skiprows=5)
lrz_demand_2016<jupyter_output><empty_output><jupyter_text>Replace NaNs with predicted values.<jupyter_code>lrz_demand_2016.loc[
    lrz_demand_2016['ActualLoad (MWh)'].isna(), 'ActualLoad (MWh)'] = lrz_demand_2016['MTLF (MWh)']
lrz_demand_2016<jupyter_output><empty_output><jupyter_text>Load 2015 MISO load data.<jupyter_code>lrz_demand_2015 = pd.read_excel('20151231_dfal_hist.xls', skiprows=5)
lrz_demand_2015<jupyter_output><empty_output><jupyter_text>Create a function converts MISO file format to a data frame with sub-area load columns.<jupyter_code>miso_zones = ['LRZ1', 'LRZ2_7', 'LRZ3_5', 'LRZ4', 'LRZ6', 'LRZ8_9_10']

def create_demand_profile(data, miso_zones):
    demand = pd.DataFrame(data[data['LoadResource Zone'] == miso_zones[0]]['ActualLoad (MWh)'])
    for z in range(1, len(miso_zones)):
        demand[miso_zones[z]] = data[data['LoadResource Zone'] == miso_zones[z]]['ActualLoad (MWh)'].to_list()
    demand.columns = miso_zones
    return demand<jupyter_output><empty_output><jupyter_text>Create sub-areas data frame for 2015 demand data<jupyter_code>miso_loadzones_2015 = create_demand_profile(lrz_demand_2015, miso_zones)
miso_loadzones_2015<jupyter_output><empty_output><jupyter_text>Set the 2015 demand date-time to be the next hour in the Eastern Timezone.<jupyter_code>start_date = pd.to_datetime('2015-01-01 01:00:00')
end_date = pd.to_datetime('2016-01-01 00:00:00')
time_interval = pd.date_range(start_date, end_date, tz="US/Eastern", freq='H')
miso_loadzones_2015.index = time_interval
miso_loadzones_2015<jupyter_output><empty_output><jupyter_text>Perform the same operation for 2016 data.<jupyter_code>miso_loadzones_2016 = create_demand_profile(lrz_demand_2016, miso_zones)
miso_loadzones_2016<jupyter_output><empty_output><jupyter_text>Set the 2016 demand date-time to be the next hour in the Eastern Timezone.<jupyter_code>start_date = pd.to_datetime('2016-01-01 01:00:00')
end_date = pd.to_datetime('2017-01-01 00:00:00')
time_interval = pd.date_range(start_date, end_date, tz="US/Eastern", freq='H')
miso_loadzones_2016.index = time_interval
miso_loadzones_2016<jupyter_output><empty_output><jupyter_text>Concatenate 2015 and 2016 together, shift date-time to UTC and then select 2016 data.<jupyter_code>miso_demand = pd.concat([miso_loadzones_2015, miso_loadzones_2016]).tz_convert('utc')

start = pd.to_datetime('2016-01-01 00:00:00')
end = pd.to_datetime('2016-12-31 23:00:00')
miso_demand = miso_demand.loc[start : end]
miso_demand
miso_demand.to_csv('miso_subarea_demand_profile.csv')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11867642379891823, 'fraction_numerical': 0.07604199809099586, 'mean_word_length': 8.527272727272727, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 41, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '560316', 'license': 'MIT', 'n_tokens_mistral': 1391, 'n_tokens_neox': 1183, 'n_words': 203, 'path': 'prereise/gather/demanddata/eia/demo/eastern_demand_v6_demo/miso/generate_miso_subarea_profile.ipynb', 'repo_name': 'lanesmith/PreREISE', 'size': '3143'}