<jupyter_start><jupyter_text>Read in data and set up dataframes<jupyter_code>df = pd.read_csv('data/clean_data.csv', index_col=0)
df.date = pd.to_datetime(df.date)
df = df[['date','close','symbol']]
df_slv = df.loc[df.symbol == 'SLV']
df_sil = df.loc[df.symbol == 'SIL']
df_gld = df.loc[df.symbol == 'GLD']
df_gdx = df.loc[df.symbol == 'GDX']
df_dji = df.loc[df.symbol == 'DJI']<jupyter_output><empty_output><jupyter_text>Silver Visualizations  Silver Bullion (SLV) Average Close by Month<jupyter_code>df_slv_mo = df_slv.groupby(pd.Grouper(key='date', freq='M')).mean()
df_slv_mo.plot()
plt.title('Avg Close of SLV by Month')
plt.ylabel('Dollars')
plt.xlabel('Time')
plt.show()<jupyter_output><empty_output><jupyter_text>Silver Bullion (SLV) and Silver Mining (SIL) Average Close by Year<jupyter_code>df_sil_mo = df_sil.groupby(pd.Grouper(key='date', freq='Y')).mean()
df_silvers = df_slv_mo.join(df_sil_mo,how='inner',on='date',lsuffix='_sil',rsuffix='_slv')
df_silvers.plot()
plt.title('Avg Close of SIL and SLV')
plt.ylabel('Price ($)')
plt.xlabel('Time')
plt.show()<jupyter_output><empty_output><jupyter_text>Silver Bullion (SLV) and Silver Mining (SIL) Average Percent Change<jupyter_code>df_silvers.pct_change().plot()
plt.title('Pct Change in Close of SIL and SLV')
plt.show()
df_dji_yr = df_dji.groupby(pd.Grouper(key='date', freq='Y')).mean()
df_dji_mo = df_dji.groupby(pd.Grouper(key='date', freq='M')).mean()
df_dji_yr.head();<jupyter_output><empty_output><jupyter_text>Silver Bullion (SLV), Silver Mining (SIL), and Market Index Average Percent Change<jupyter_code>df_sv_ind = df_silvers.join(df_dji_mo, how='left',on='date')
df_sv_ind.pct_change().plot()
plt.title('Pct Change in Close of SIL, SLV, and Market Index')
plt.show()<jupyter_output><empty_output><jupyter_text>Gold Visualizations  Gold Bullion (GLD) and Gold Mining (GDX) Average Close by Year<jupyter_code>df_gld_yr = df_gld.groupby(pd.Grouper(key='date', freq='Y')).mean()
df_gdx_yr = df_gdx.groupby(pd.Grouper(key='date', freq='Y')).mean()
df_golds_yr = df_gld_yr.join(df_gdx_yr, how='left',on='date',lsuffix='_gld',rsuffix='_gdx')
df_gd_ind_yr = df_golds_yr.join(df_dji_yr,how='left',on='date')
df_golds_yr.plot()
plt.title('Avg Close of GLD and GDX by Year')
df_gld_mo = df_gld.groupby(pd.Grouper(key='date', freq='M')).mean()
df_gdx_mo = df_gdx.groupby(pd.Grouper(key='date', freq='M')).mean()
df_golds_mo = df_gld_mo.join(df_gdx_mo, how='left',on='date',lsuffix='_gld',rsuffix='_gdx')
df_gd_ind_mo = df_golds_mo.join(df_dji_mo,how='left',on='date')
df_golds_mo.plot()
plt.title('Avg Close of GLD and GDX by Month')<jupyter_output><empty_output><jupyter_text>Gold Bullion (GLD), Gold Mining (GDX), and Market Index Average Percent Change by Year<jupyter_code>df_gd_ind_yr.pct_change().plot()
plt.title('Pct Change of GLD, GDX, and Market Index by Year')
plt.show()
df_gd_ind_mo.pct_change().plot()
plt.title('Pct Change of GLD, GDX, and Market Index by Month')
plt.show()<jupyter_output><empty_output><jupyter_text>Step 1: Model Identification  ARIMA Model using Box-Jenkins method  Is there seasonality?<jupyter_code># look at an autocorrelation plot with a fixed lag of 5 
plt.figure(figsize=(10,10))
lag_plot(df_gdx_mo, lag=5)
plt.title('Multi-Year GDX Autocorrelation plot')
plt.show()
autocorrelation_plot(df_gdx_mo)
plt.show()
plt.figure(figsize=(10,10))
plt.plot(df_gdx_mo)
plt.xlabel('Months')
plt.ylabel('Price')
plt.title('Multi-Year GDX Close Price by Month')
plt.show()
# create dataframe with one year's data 
df_gdx_mo_2019 = df_gdx_mo.loc[(df_gdx_mo.index> '2018-09-01') & (df_gdx_mo.index < '2019-09-01')]
df_gdx_mo_2019.shape
plt.plot(df_gdx_mo_2019)
plt.title('2019 GDX Price Line Plot')
plt.ylabel('Price')
plt.xlabel('Month')
plt.show()
lag_plot(df_gdx_mo_2019, lag=5)
plt.title('2019 GDX Price Lag Plot')
plt.show()
autocorrelation_plot(df_gdx_mo_2019)
plt.title('2019 GDX Price Autocorrelation Plot')
plt.show()
df_gdx_mo_2019.plot(kind='kde')
plt.title('2019 GDX Price Density Plot')
plt.xlabel('Price')
plt.show()<jupyter_output><empty_output><jupyter_text>Three main parameters: p = number of lag observations d = degree of differencing q = the size of the moving average window<jupyter_code>model = ARIMA(df_gdx_mo, order=(5,1,0))
model_fit = model.fit(disp=0)
print(model_fit.summary())<jupyter_output>ARIMA Model Results                              
==============================================================================
Dep. Variable:                D.close   No. Observations:                  161
Model:                 ARIMA(5, 1, 0)   Log Likelihood                -386.619
Method:                       css-mle   S.D. of innovations              2.670
Date:                Wed, 09 Oct 2019   AIC                            787.238
Time:                        11:57:30   BIC                            808.807
Sample:                    06-30-2006   HQIC                           795.996
                         - 10-31-2019                                         
=================================================================================
                    coef    std err          z      P>|z|      [0.025      0.975]
---------------------------------------------------------------------------------
const            -0.0680      0.259     -0.[...]<jupyter_text>One-Way Anova  ANOVA stands for analysis of variance.   If the null hypothesis is not rejected, the F-statistic follows the F-distribution with $\alpha - 1 $ and $N - \alpha$ degrees of freedom.   Assumptions:All populations have common variance Within each sample observations are sampled randomly and independent of each other Each group sample is drawn from normally distributed population Factor effects are additive<jupyter_code>df.head()
df_recent = df.loc[(df['date'] > '2019-08-01') & (df['symbol'] != 'DJI') & (df['symbol'] != 'GLD')]
df_recent.shape
df_recent.groupby('symbol').describe()<jupyter_output><empty_output><jupyter_text>Using $\alpha = 0.05$ $H_{o}: \mu_{1} = \mu_{2} = \mu_{3} $ $H_{a}: $ At least one of the means differs from the other means<jupyter_code>formula = 'close ~ symbol'
lm = ols(formula, df_recent).fit()
table = sm.stats.anova_lm(lm, typ=2)
print(table)
df_recent.boxplot('close', by='symbol')
plt.xlabel('Exchange-Traded Fund')
plt.ylabel('Price')
plt.title('Mean GDX, SIL, and SLV Price since August 1, 2019')
plt.style.use('seaborn-poster')
plt.suptitle('')
plt.savefig('boxplot_al.png')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.17931248064416228, 'fraction_numerical': 0.026943326107153918, 'mean_word_length': 4.311677631578948, 'pattern_counts': {'":': 0, '<': 45, '<?xml version=': 0, '>': 47, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '267788', 'license': 'MIT', 'n_tokens_mistral': 2557, 'n_tokens_neox': 2356, 'n_words': 480, 'path': 'al-ntbk-v3.ipynb', 'repo_name': 'akharina/gold_silver_etf', 'size': '6458'}