<jupyter_start><jupyter_text>Time Series - Lecture 2: Lags, Differencing, AR, and MAThe goal of time series is similar to regressions in that we are trying to separate **trends/patterns/model** from **resisduals/noise**. The difference is that time is a useful component in finding trends/patterns.Today, we will study **how** time variable is used in finding patterns. Agenda1. Lags and Differencing2. Autoregression (AR)3. Rolling Window or moving average (MA)Let us use the same example to explore these concepts:<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pylab as plt
df = pd.read_csv('data/multiTimeline.csv', skiprows=1)
df.columns = ['month', 'diet', 'gym', 'finance']
df.month = pd.to_datetime(df.month)
df.set_index('month', inplace=True)
df.plot(figsize=(9,9), linewidth=2, fontsize=15)<jupyter_output><empty_output><jupyter_text>Lags and Differencing Lags- Normally, "lag of $X_t$" is denoted $LX_t = X_{t-1}$, $L^2X_t=X_{t-2}$, and etc. - In data, usually create lags by shifting index by one, and normally you loose one data point<jupyter_code>df.diet
df.diet.shift(-1)
df['diet1'] = df.diet.shift(-1)
pd.plotting.lag_plot(df.diet,lag=1)
plt.scatter(df['diet'],df['diet1'])<jupyter_output><empty_output><jupyter_text>Differencing- Change between consecutive observations, and can be written as: $$y'_t=y_t-y_{t-1}$$- Second -order differencing: $$ \begin{align} y''_t & = y'_t-y'_{t-1} \\ & = (y_t-y_{t-1}) - (y_{t-1}-y_{t-2}) \end{align} $$- Seasonal (or "lag-m differencing"): $$y'_t=y_t-y_{t-m}$$ where $m$ is the period for a season<jupyter_code>pd.merge(diet, diet.diff(), left_index=True, right_index=True)
pd.merge(diet, diet.diff(periods=2), left_index=True, right_index=True)<jupyter_output><empty_output><jupyter_text>- Differencing can eliminates (or reduces) trend and seasonality<jupyter_code>df['diet'].plot()
diet.diff().plot()<jupyter_output><empty_output><jupyter_text>See that you have removed much of the trend and you can really see the peaks in January every year. Note: You can also perform additional data manipulations (e.g., 2nd order differencing) if the trend is not yet entirely removed. See [here](https://otexts.com/fpp2/stationarity.html) for more on differencing.Differencing can be helpful in turning your time series into a stationary time series. > Stationarity: properties of the data generating process do not depend on the time at which the series is observed.Stationary time series are useful because many time series forecasting methods are based on the assumption that the time series is approximately stationary.More noticeable example can be found in asset prices. For example, let's look into 200 consecurive days of Google stock price and differenced value.<jupyter_code>from PIL import Image
Image.open("images/GOOGdiff.png")<jupyter_output><empty_output><jupyter_text>Autoregression- The model predicts the variable of interest using a linear combination of **historical** values of the variable. - An **AR(p) model** is written as:$$y_t = \beta_0 + \beta_1y_{t-1} + \beta_2y_{t-2} + \cdots + \beta_py_{t-p}+\epsilon_t$$For an AR(1) model, when:- $\beta_1=0$, the model, or $y_t$ is equivalent to noise- $\beta_1=1$ and $\beta_0=0$, the model, or $y_t$ is equivalent to random walk - $\beta_1=1$ and $\beta_0 \neq 0$, the model, or $y_t$ is equivalent to random walk with a drift - $\beta_1<0$, $y_t$ tends to oscillate around the meanWe normally use autoregressive models for the following stationary cases:- for an AR(1) model, $-1<\beta_1<1$- for an AR(2) model, $-1<\beta_2<1,\beta_1+\beta_2<1,\beta_2-\beta_1<1$when $p\geq3$, the conditions are more complicated.<jupyter_code>df.index = pd.DatetimeIndex(df.index).to_period('M')
from statsmodels.tsa.arima_model import ARMA
model = ARMA(df.diet, order=(1,0))
result = model.fit()
print(result.summary())
result.plot_predict(start='2004-01-01',end='2020-08-01')
plt.show()
df.index<jupyter_output><empty_output><jupyter_text>Moving Average (or Rolling Window)- Rather than using the past values, a moving average model uses past **errors** in a regression.- An **MA(q) model** is written as: $$y_t=\beta_0+\epsilon_t+\beta_1\epsilon_{t-1}+\cdots+\beta_q\epsilon_{t-q}$$We normally use moving average models for the following (invertibility) cases:- for an MA(1) model, $-1<\beta_1<1$- for an MA(2) model, $-1-1,\beta_1-\beta_2<1$when $p\geq3$, the conditions are more complicated.<jupyter_code>model = ARMA(df.diet, order=(0,7))
result = model.fit()
print(result.summary())
result.plot_predict(start='2004-01-01',end='2020-08-01')
plt.show()<jupyter_output>ARMA Model Results                              
==============================================================================
Dep. Variable:                   diet   No. Observations:                  200
Model:                     ARMA(0, 7)   Log Likelihood                -628.329
Method:                       css-mle   S.D. of innovations              5.492
Date:                Sun, 20 Sep 2020   AIC                           1274.658
Time:                        21:16:43   BIC                           1304.343
Sample:                    01-31-2004   HQIC                          1286.671
                         - 08-31-2020                                         
==============================================================================
                 coef    std err          z      P>|z|      [0.025      0.975]
------------------------------------------------------------------------------
const         50.3819      1.709     29.483      0.0[...]
{'fraction_non_alphanumeric': 0.16218155722999641, 'fraction_numerical': 0.03606027987082885, 'mean_word_length': 3.46, 'pattern_counts': {'":': 0, '<': 38, '<?xml version=': 0, '>': 30, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '265688', 'license': 'MIT', 'n_tokens_mistral': 1857, 'n_tokens_neox': 1729, 'n_words': 532, 'path': 'Time Series Analysis - Lecture 2.ipynb', 'repo_name': 'doublebyte1/Time-Series-Analysis', 'size': '5574'}