<jupyter_start><jupyter_text>Step 4 – Task 2. Time Series Forecasting - CLASS ASSIGNMENT<jupyter_code>!pip install --upgrade pip
!pip install numpy --upgrade
!pip install statsmodels --upgrade
!pip install sklearn<jupyter_output>Requirement already satisfied: sklearn in /Users/asathi/anaconda3/lib/python3.7/site-packages (0.0)
Requirement already satisfied: scikit-learn in /Users/asathi/anaconda3/lib/python3.7/site-packages (from sklearn) (0.19.2)<jupyter_text>import libraries<jupyter_code>import pandas as pd

from datetime import date
from datetime import timedelta

import matplotlib.pyplot as plt

from statsmodels.tsa.seasonal import seasonal_decompose
from statsmodels.tsa.holtwinters import Holt

from sklearn.metrics import mean_squared_error

import warnings
warnings.filterwarnings('ignore')

%matplotlib inline<jupyter_output><empty_output><jupyter_text>Set up environment and connect to Google Drive<jupyter_code>using_Google_colab = False
using_Anaconda_on_Mac_or_Linux = True
using_Anaconda_on_windows = False
if using_Google_colab:
    from google.colab import drive
    drive.mount('/content/drive')<jupyter_output><empty_output><jupyter_text>DM2.1 Open Notebook  and upload the state Analytics Base Table, select incremental cases / deaths for NJ and normalize - Activity 1<jupyter_code>if using_Google_colab:             
    df_abt_by_state = pd.read_csv('/content/drive/MyDrive/COVID_Project/output/abt_by_state.csv',
                                  index_col=['Date'], parse_dates=['Date'])
if using_Anaconda_on_Mac_or_Linux:
    df_abt_by_state = pd.read_csv('../output/abt_by_state.csv',
                                  index_col=['Date'], parse_dates=['Date'])
if using_Anaconda_on_windows:
    df_abt_by_state = pd.read_csv('..\output\abt_by_state.csv',
                                  index_col=['Date'], parse_dates=['Date'])
df_abt_by_state<jupyter_output><empty_output><jupyter_text>Choose state - NJ<jupyter_code>df_abt_chosen_state = df_abt_by_state[df_abt_by_state['State'] == 'NJ']
df_abt_chosen_state = df_abt_chosen_state.dropna()
df_abt_chosen_state<jupyter_output><empty_output><jupyter_text>DM2.2 Activity 2 - Compute Weekly Summaries<jupyter_code>df_abt_chosen_state_weekly = df_abt_chosen_state.resample('W').sum()
df_abt_chosen_state_weekly<jupyter_output><empty_output><jupyter_text>Normalize data<jupyter_code>state_covid_cases = df_abt_chosen_state['Incremental Cases']
cases_max = state_covid_cases.max()
state_covid_cases = state_covid_cases/cases_max
state_covid_deaths = df_abt_chosen_state['Incremental Deaths']
deaths_max = state_covid_deaths.max()
state_covid_deaths = state_covid_deaths / deaths_max<jupyter_output><empty_output><jupyter_text>DM2.3 - Activity 3 - Compute seasonal_decompose Seasonal decomposition using moving averages.  It returns A object with seasonal, trend, and residual attributes.(x, model='additive', filt=None, period=None, two_sided=True, extrapolate_trend=0)Attributes:- x Time series. - model{“additive”, “multiplicative”}, (optional) Type of seasonal component. - filt array_like, (optional) The filter coefficients for filtering out the seasonal component. - period int, (optional) Period of the series. - two_sidedbool, (optional) The moving average method used in filtering. If True (default), a centered moving average is computed using the filt. If False, the filter coefficients are for past values only.- extrapolate_trendint or ‘freq’, optional If set to > 0, the trend resulting from the convolution is linear least-squares extrapolated on both ends (or the single one if two_sided is False) considering this many (+1) closest points.<jupyter_code>sd =seasonal_decompose(state_covid_cases,freq=7)
sd.plot()<jupyter_output><empty_output><jupyter_text>Use Trend Data<jupyter_code>plt.figure(figsize=(17,8))
plt.title('Seasonal Decomposition Trend and Seasonality')
plt.ylabel('Incremental Cases')
plt.xlabel('Date')
plt.plot(sd.trend, 'c', label='Trend')
plt.plot(sd.seasonal - 0.2, 'r', label="Seasonal Variations")
plt.plot(sd.resid, 'b', label='Residuals')
plt.legend(loc='best')
plt.grid(True)
plt.show()<jupyter_output><empty_output><jupyter_text>DM2.4 - Apply Holt’s algorithm to training and test sets  Divide into training and test sets<jupyter_code>train = state_covid_cases[:'2020-11-1']
test  = state_covid_cases['2020-11-2':]
test<jupyter_output><empty_output><jupyter_text>Forecast using Holt algorithm various options:- optimized- Linear trend- linear trend with dampening  If you are using issues using plot as a method with ax, then please try plt.plot function<jupyter_code>forecast_des =  Holt(train).fit(optimized=True)
forecast_des_1 = Holt(train).fit(smoothing_level=0.9, smoothing_slope=0.9, optimized=False)
forecast_des_2 = Holt(train, damped=True).fit(smoothing_level=1, smoothing_slope=1)
des_fcst = forecast_des.forecast(30).rename('Forecast')
des_fcst_1 = forecast_des_1.forecast(30).rename("Holt's linear trend")
des_fcst_2 = forecast_des_2.forecast(30).rename("damped trend")
#ax = test.plot(color="black",  figsize=(17,8), legend=True)
#des_fcst.plot(ax=ax, color='blue',legend=True)
#des_fcst_1.plot(ax=ax, color ="red", legend=True)
#des_fcst_2.plot(ax=ax, color='cyan', legend=True)
plt.plot(test, 'black', label='Test')
plt.plot(des_fcst, 'blue', label='Test')
plt.plot(des_fcst_1, 'red', label='Test')
plt.plot(des_fcst_2, 'cyan', label='Test')
plt.title("Double exponential smoothing")
plt.show()<jupyter_output><empty_output><jupyter_text>DM2.5 - Compute seasonal decompose at weekly level - CLASS ASSIGNMENT<jupyter_code>df_abt_chosen_state_weekly<jupyter_output><empty_output><jupyter_text>Normalize data<jupyter_code>state_covid_cases_weekly = df_abt_chosen_state_weekly['Incremental Cases']
cases_weekly_max = state_covid_cases_weekly.max()
state_covid_cases_weekly = state_covid_cases_weekly/cases_max
state_covid_deaths_weekly = df_abt_chosen_state_weekly['Incremental Deaths']
deaths_weekly_max = state_covid_deaths_weekly.max()
state_covid_deaths_weekly = state_covid_deaths_weekly / deaths_weekly_max<jupyter_output><empty_output><jupyter_text>Compute Seasonal Decompose<jupyter_code>sd =seasonal_decompose(state_covid_cases_weekly,freq=7)
sd.plot()
sd =seasonal_decompose(state_covid_deaths_weekly,freq=7)
sd.plot()<jupyter_output><empty_output><jupyter_text>DM2.6 - Compute Holt model at weekly level - CLASS ASSIGNMENT  Create Training and test data set<jupyter_code>train_weekly = state_covid_cases_weekly[:'2020-11-1']
test_weekly  = state_covid_cases_weekly['2020-11-2':]<jupyter_output><empty_output><jupyter_text>Forecast using Holt method<jupyter_code>forecast_des =  Holt(train_weekly).fit(optimized=True)
forecast_des_1 = Holt(train_weekly).fit(smoothing_level=0.9, smoothing_slope=0.9, optimized=False)
forecast_des_2 = Holt(train_weekly, damped=True).fit(smoothing_level=1, smoothing_slope=1)
des_fcst = forecast_des.forecast(30).rename('Forecast')
des_fcst_1 = forecast_des_1.forecast(30).rename("Holt's linear trend")
des_fcst_2 = forecast_des_2.forecast(30).rename("damped trend")
plt.plot(test_weekly, 'black', label='Test')
plt.plot(des_fcst, 'blue', label='Test')
plt.plot(des_fcst_1, 'red', label='Test')
plt.plot(des_fcst_2, 'cyan', label='Test')
plt.title("Double exponential smoothing")
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11153740750890655, 'fraction_numerical': 0.015483694162784325, 'mean_word_length': 7.457705677867903, 'pattern_counts': {'":': 0, '<': 64, '<?xml version=': 0, '>': 65, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '257543', 'license': 'MIT', 'n_tokens_mistral': 2745, 'n_tokens_neox': 2587, 'n_words': 480, 'path': 'Notebook-Class-Assignment-Answers/Step-4-Develop-Model-Task-2-Forecasting-Class-Assignment.ipynb', 'repo_name': 'drarvindsathi/covid_project', 'size': '7298'}