<jupyter_start><jupyter_text>Before I imported the CSV, I had to edit it slightly so it would work (and because there were some 60 or so columns)<jupyter_code>weather_1 = pd.read_csv('WeatherData/weatherdata_1_e.csv', low_memory=False , na_values=[0.0])
weather_2 = pd.read_csv('WeatherData/weatherdata_2_e.csv', low_memory=False, na_values=[0.0])
weather_1.head()
weather_2.head()
weather_d = pd.merge(weather_2, weather_1, how='outer')
weather_d.head(20)<jupyter_output><empty_output><jupyter_text>Now, lets look at the data we want to know,Say, What month on average is the hottest!What month on average recieves the most rainLet's plot average monthly rain over the years as welland plot Average Monthly temp over the years.Average Wind per monthPlot wind over the years! Part 1: TEMPERATURE,<jupyter_code>weather_d.count()<jupyter_output><empty_output><jupyter_text>For this, we will use Hourly Dry Bulb Temperature for simplicity, since this is the closest thing to a standard temperature<jupyter_code>tempdf = weather_d[pd.notnull(weather_d['HourlyDryBulbTemperature'])]
tempdf.count()<jupyter_output><empty_output><jupyter_text>It looks like there is infact 100% coverage of the dry bulb temperature<jupyter_code>#What month on average is the hottest!
tempdf.index = pd.to_datetime(tempdf['DATE'])
tempdf.index.hour[:50]
tempdf.head(10)
import seaborn as sns; 
sns.set(font_scale=1.5)
sns.set_style("whitegrid")
fig, ax = plt.subplots(figsize = (10,10))

temprf = tempdf[:50]
# add the x-axis and the y-axis to the plot
ax.plot(temprf['DATE'], 
        temprf['HourlyDryBulbTemperature'], 
        color = 'red')

# rotate tick labels
plt.setp(ax.get_xticklabels(), rotation=45)

# set title and labels for axes
ax.set(xlabel="Date",
       ylabel="Temperature",
       title="Temperature \n Seattle");
tempdf.loc['2014-01-20':'2014-01-22']
import seaborn as sns
sns.set(rc={'figure.figsize':(11, 4)})
temperature_data = tempdf[['HourlyDryBulbTemperature']]
temperature_data.head()
temperature_data.index
plt.plot_date(temperature_data.index, temperature_data['HourlyDryBulbTemperature'])
fig, ax = plt.subplots()
ax.plot(temperature_data.index,temperature_data['HourlyDryBulbTemperature'])
temperature_data['HourlyDryBulbTemperature'] = temperature_data.apply(pd.to_numeric, errors='coerce')
temp_data_monthly = temperature_data.resample('M').mean()
temp_data_monthly.head()
temp_data_monthly.tail()
fig, ax = plt.subplots()
ax.plot(temp_data_monthly, marker='', linestyle='-', linewidth=0.5)
ax.set_ylabel('Temperature')
ax.set_xlabel('Monthly/Yearly Time')<jupyter_output><empty_output><jupyter_text>Now for precipitaiton/rain<jupyter_code>tempdf.head(10)
precipitation_data = tempdf[['HourlyPrecipitation']].fillna(0)
precipitation_data.head()
precipitation_data['HourlyPrecipitation'] = precipitation_data.apply(pd.to_numeric, errors='coerce')
prec_data_monthly = precipitation_data.resample('M').sum()
prec_data_monthly.head()
fig, ax = plt.subplots()
ax.plot(prec_data_monthly, marker='', linestyle='-', linewidth=0.5)
ax.set_ylabel('Precipitation Monthly Sum')
ax.set_xlabel('Monthly/Yearly Time')<jupyter_output><empty_output><jupyter_text>Now lets break it down by year!<jupyter_code>#remake but instead of monthly, lets do weekly
prec_data_monthly2 = precipitation_data.resample('M').sum()

prec_data_monthly2['month'] = prec_data_monthly2.index.month
print(prec_data_monthly2.head())
data1 = prec_data_monthly2['2005-04-30':'2005-12-31']
data1.set_index(data1['month'])

data2 = prec_data_monthly2['2006-01-01':'2006-12-31']
data2.set_index(data2['month'])

data3 = prec_data_monthly2['2007-01-01':'2007-12-31']
data3.set_index(data3['month'])

data4 = prec_data_monthly2['2008-01-01':'2008-12-31']
data4.set_index(data4['month'])


data5 = prec_data_monthly2['2009-01-01':'2009-12-31']
data5.set_index(data5['month'])


data6 = prec_data_monthly2['2010-01-01':'2010-12-31']
data6.set_index(data6['month'])


data7 = prec_data_monthly2['2011-01-01':'2011-12-31']
data7.set_index(data7['month'])

data8 = prec_data_monthly2['2012-01-01':'2012-12-31']
data8.set_index(data8['month'])

data9 = prec_data_monthly2['2013-01-01':'2013-12-31']
data9.set_index(data9['month'])

data10 = prec_data_monthly2['2014-01-01':'2014-12-31']
data10.set_index(data10['month'])

data11 = prec_data_monthly2['2015-01-01':'2015-12-31']
data11.set_index(data11['month'])

data12 = prec_data_monthly2['2016-01-01':'2016-12-31']
data12.set_index(data12['month'])

data13 = prec_data_monthly2['2017-01-01':'2017-12-31']
data13.set_index(data13['month'])

data14 = prec_data_monthly2['2018-01-01':'2018-12-31']
data14.set_index(data14['month'])

data15 = prec_data_monthly2['2019-01-01':'2019-12-31']
data15.set_index(data15['month'])

fig, ax = plt.subplots()
ax.plot(data1['month'], data1['HourlyPrecipitation'], marker='', linestyle='-', linewidth=1)
ax.plot(data2['month'], data2['HourlyPrecipitation'], marker='', linestyle='-', linewidth=1)
ax.plot(data3['month'], data3['HourlyPrecipitation'], marker='', linestyle='-', linewidth=1)
ax.plot(data4['month'], data4['HourlyPrecipitation'], marker='', linestyle='-', linewidth=1)
ax.plot(data5['month'], data5['HourlyPrecipitation'], marker='', linestyle='-', linewidth=1)
ax.plot(data6['month'], data6['HourlyPrecipitation'], marker='', linestyle='-', linewidth=1)
ax.plot(data7['month'], data7['HourlyPrecipitation'], marker='', linestyle='-', linewidth=1)
ax.plot(data8['month'], data8['HourlyPrecipitation'], marker='', linestyle='-', linewidth=1)
ax.plot(data9['month'], data9['HourlyPrecipitation'], marker='', linestyle='-', linewidth=1)
ax.plot(data10['month'], data10['HourlyPrecipitation'], marker='', linestyle='-', linewidth=1)
ax.plot(data11['month'], data11['HourlyPrecipitation'], marker='', linestyle='-', linewidth=1)
ax.plot(data12['month'], data12['HourlyPrecipitation'], marker='', linestyle='-', linewidth=1)
ax.plot(data13['month'], data13['HourlyPrecipitation'], marker='', linestyle='-', linewidth=1)
ax.plot(data14['month'], data14['HourlyPrecipitation'], marker='', linestyle='-', linewidth=1)
ax.plot(data11['month'], data11['HourlyPrecipitation'], marker='', linestyle='-', linewidth=1)
ax.set_ylabel('Precipitation Sum per month')
ax.set_xlabel('Month')<jupyter_output><empty_output><jupyter_text>Now we need to save the TempDF frame out to things we'll use when trying to find a correlation<jupyter_code>tempdf.head()
tempdf.to_pickle("WeatherData/weather.pkl")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1694495980210266, 'fraction_numerical': 0.0678726035868893, 'mean_word_length': 8.499265785609397, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '231896', 'license': 'MIT', 'n_tokens_mistral': 2677, 'n_tokens_neox': 2381, 'n_words': 394, 'path': 'SeattleWeatherAnalysis.ipynb', 'repo_name': 'nullmage/DataMiningProject', 'size': '6468'}