<jupyter_start><jupyter_text>Blog Post Draft Shuo and Jacob Summary: are wheat prices being driven by a hot, dry summer?Outline1. Motivate with graphs of futures and spot prices2. Show weather data (temperature and precipitation for weather stations in wheat-growing areas)3. Compare weather data with wheat prices, see if they matchTo do:Compile weather data, compare weather this year to weather from past years A recent Wall Street Journal article highlighted increasing futures prices for wheat and suggested that these were in part due to "extreme dryness and heat in the world’s breadbaskets, including Russia and the U.S." ([link to full article](https://www.wsj.com/articles/wheat-prices-skyrocket-after-dry-weather-coronavirus-lockdowns-11602249791?st=14rd570yuifxif8)). In this blog post, we explore the extent to which this year's prices appear to be driven by weather. Instead of sophisticated econometric analyses, we compare this year's weather in US wheat-growing areas to that of the past 10 years. The measures of weather that we use are precipitation and Extreme Degree Days (defined later). This post proceeds as follows. We review futures prices for wheat, highlighting a significant increase from Aug to Oct 2020. We also look at the amount of wheat in storage in 2020 compared to previous years and find that it has decreased somewhat, perhaps playing some role in the increased prices. Finally, we consider the weather data. Extreme Degree Days are found to have notably increased in several wheat-growing regions, and precipitation has also decreased this year. These weather conditions can lead to significantly lower wheat yields, driving up wheat prices.<jupyter_code># Basics
import os
import pandas as pd
import numpy as np
import random as rnd
from datetime import datetime

# Visualization
import seaborn as sns
import matplotlib.pyplot as plt
import matplotlib
from mpl_toolkits import mplot3d
%matplotlib inline

# Checking
import IPython<jupyter_output><empty_output><jupyter_text>1. Futures price<jupyter_code>cwd = os.getcwd()

# Read in prices data

filename = 'Futures_price_Wheat_2020_Dec.csv'
location = os.path.join(cwd, filename)
futures_df = pd.read_csv(location, header=0)
futures_df['Date'] = [datetime.strptime(x, '%m/%d/%Y').date() if x.find('-')==-1 \
                      else datetime.strptime(x, '%Y-%m-%d').date() \
                      for x in futures_df['Date']]
futures_df = futures_df[futures_df['Date']>datetime(2020,1,1).date()]
fig, ax = plt.subplots(figsize=(15,5))
sns.lineplot(data=futures_df, \
             x="Date", \
             y="Close", \
             hue="Contract").set_title("Futures Prices (2020 December Contract)")
fig.savefig('futures.png')<jupyter_output><empty_output><jupyter_text>Here we see that the winter wheat futures price have increased substantially since from the late summer to fall, and the one for spring wheat also increased mildly. COVID has had major impacts on both agricultural producers and consumers, but it's difficult to know to what degree these futures prices have been driven by the pandemic. Still, if you look at this graph, there is a big increase in futures price from mid-March to the beginning of April, which is when shelter-in-place orders were issued in the US. You might also notice another increase in prices starting in July, which is when the US experienced a "second wave" of COVID cases (see this [CDC graph](https://covid.cdc.gov/covid-data-tracker/trends_dailytrendscases)). Of course, these correlations could be spurious, but the matching is interesting to note.But what about the huge increase from August to October? What has driven that? Could it be weather, like the WSJ article suggested. Possibly. But before getting to this year's wheat weather (say that 5 times fast), we consider the role of storage. Could low storage quantities have driven up recent futures prices?  2. Storage<jupyter_code># Read in stock data

filename = 'Wheat_stock.xlsx'
location = os.path.join(cwd, filename)
stock_df = pd.read_excel(location, header=1)
stock_df['Year'] = [x.year for x in stock_df['Report_date']]
stock_df['Report Date Each Year'] = [x.month for x in stock_df['Report_date']]
numtostr = {"Report Date Each Year": {3: 'March 1st', 6: 'June 1st', 9: 'September 1st', 12: 'December 1st'}}
stock_df.replace(numtostr, inplace=True)
fig, ax = plt.subplots(figsize=(15,5))
sns.barplot(x="Report Date Each Year", y="Total", hue="Year", data=stock_df, color='darkblue').set_title("Total Stock Reported from 2016 to 2020 (1,000 Bushels)")
fig.savefig('stock.png')<jupyter_output><empty_output><jupyter_text>Storage quantities are a bit lower this year, but it's hard to know if the decrease is significant enough to increase prices. There's nothing in this graph that jumps out at us. Let's move on to weather conditions.  3. Temperature We start with the measure of Extreme Degree Days (EDD) this year compared to recent years. What is EDD? The formula looks bad ($\sum_d max(T_d - 30,0)$) but it's a count of the number of days with temperatures above 30$^{\circ}$ Celsius (86$^{\circ}$ F) but where 31 degree days count once, 32 degree days count twice, etc. It should be a nice way of summarizing the number of "hot" days. Here, we graph EDD from this year compared to the average EDD from 2010-2019. FIGURE<jupyter_code># Read in EDD data
# Winter data
cwd = os.getcwd()
filename = 'EDD_winter.csv'
location1 = os.path.join(cwd, filename)
EDDwinter_df = pd.read_csv(location1, header=0)
# Spring data
filename = 'EDD_spring.csv'
location2 = os.path.join(cwd, filename)
EDDspring_df = pd.read_csv(location2, header=0)
EDDspring_df = EDDspring_df.drop(['Year'], axis=1) # Drop the Year column
# Transpose it, take mean of 2010-2019, name columns, drop old columns
EDDwinter_df = EDDwinter_df.transpose()
EDDwinter_df['Mean EDD 2010-2019']  = EDDwinter_df[[0,1,2,3,4,5,6,7,8,9]].mean(axis=1)
EDDwinter_df['EDD 2020'] = EDDwinter_df[[10]]
EDDwinter_df = EDDwinter_df.drop([0,1,2,3,4,5,6,7,8,9,10], axis=1)
EDDwinter_df['City'] = EDDwinter_df.index
# Do everything again for spring wheat
EDDspring_df = EDDspring_df.transpose()
EDDspring_df['Mean EDD 2010-2019']  = EDDspring_df[[0,1,2,3,4,5,6,7,8,9]].mean(axis=1)
EDDspring_df['EDD 2020'] = EDDspring_df[[10]]
EDDspring_df = EDDspring_df.drop([0,1,2,3,4,5,6,7,8,9,10], axis=1)
EDDspring_df['City'] = EDDspring_df.index
# Get the data ready (put it into lists)
x1 = list(EDDwinter_df['Mean EDD 2010-2019'])
y1 = list(EDDwinter_df['EDD 2020'])
labels1 = list(EDDwinter_df['City'])

x2 = list(EDDspring_df['Mean EDD 2010-2019'])
y2 = list(EDDspring_df['EDD 2020'])
labels2 = list(EDDspring_df['City'])

# Make figure with two subplots, winter and spring wheat
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(15,5))

# Winter plot
ax1.set_title('Winter Wheat')
ax1.set(xlabel='Mean EDD 2010-2019', ylabel='EDD 2020')
ax1.scatter(x1, y1)
for i, txt in enumerate(labels1):
    ax1.annotate(txt, (x1[i], y1[i])) # add city labels to each point
z1 = np.linspace(*ax1.get_xlim()) # these two lines draw 45 degree line
ax1.plot(z1, z1)

# Spring plot
ax2.set_title('Spring Wheat')
ax2.set(xlabel='Mean EDD 2010-2019')
ax2.scatter(x2, y2)
for i, txt in enumerate(labels2):
    ax2.annotate(txt, (x2[i], y2[i])) # add city labels to each point
z2 = np.linspace(*ax2.get_ylim()) # these two lines draw 45 degree line
ax2.plot(z2, z2)

plt.show()

fig.savefig('EDDplots.png')
# Read in stock data

filename = 'EDD_spring_figure.csv'
location = os.path.join(cwd, filename)
EDD_df = pd.read_csv(location, header=0)
fig, ax = plt.subplots(figsize=(15,5))
sns.barplot(x="Location", y="EDD", hue="Year", data=EDD_df, color='darkblue').set_title("Extreme Degree Days at 4 Locations on the Spring Wheat Belt")
fig.savefig('EDD.png')<jupyter_output><empty_output><jupyter_text>This graph suggests that there has been a marked increase in the number of EDD this year compared to past years. What role has this played in driving up the prices? We can't say for sure, but this year definitely had more warm days than past years.  4. Precipitation<jupyter_code># Read in stock data

filename = 'Precipitation_winter_wheat_belt.xlsx'
location = os.path.join(cwd, filename)
pcp_df = pd.read_excel(location, header=0)

fig, ax = plt.subplots(figsize=(15,5))
sns.lineplot(data=pcp_df, \
             x="Month", \
             y="Last year", label="Past year precipitation").set_title("Precipitation in Winter Wheat Belt (area weighted)")
sns.lineplot(data=pcp_df, \
             x="Month", \
             y="Ten-year average", label="Ten-year average precipitation (2009-2019)")
plt.legend()
ax.set(xlabel='Month', ylabel='Precipitation (Inch)')
fig.savefig('pcp_1.png')
# Read in stock data

filename = 'Precipitation_spring_wheat_belt.xlsx'
location = os.path.join(cwd, filename)
pcp_df = pd.read_excel(location, header=0)

fig, ax = plt.subplots(figsize=(15,5))
sns.lineplot(data=pcp_df, \
             x="Month", \
             y="Last year", label="Past year precipitation").set_title("Precipitation in Spring Wheat Belt (area weighted)")
sns.lineplot(data=pcp_df, \
             x="Month", \
             y="Ten-year average", label="Ten-year average precipitation (2009-2019)")
plt.legend()
ax.set(xlabel='Month', ylabel='Precipitation (Inch)')
fig.savefig('pcp_2.png')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11120625066895001, 'fraction_numerical': 0.033179920796318096, 'mean_word_length': 5.04789644012945, 'pattern_counts': {'":': 1, '<': 21, '<?xml version=': 0, '>': 22, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '533408', 'license': 'Apache-2.0', 'n_tokens_mistral': 3283, 'n_tokens_neox': 3019, 'n_words': 1045, 'path': 'Blog_figures.ipynb', 'repo_name': 'jacoblefler/ARE_231', 'size': '9343'}