<jupyter_start><jupyter_text>Checking if a pair of stocks is cointegrated  Imports<jupyter_code>import numpy as np
import pandas as pd
from sklearn.linear_model import LinearRegression
from statsmodels.tsa.stattools import adfuller
import matplotlib.pyplot as plt
import quiz_tests

# Set plotting options
%matplotlib inline
plt.rc('figure', figsize=(16, 9))
# fix random generator so it's easier to reproduce results
np.random.seed(2018)
# use returns to create a price series
drift = 100
r1 = np.random.normal(0, 1, 1000) 
s1 = pd.Series(np.cumsum(r1), name='s1') + drift

#make second series
offset = 10
noise = np.random.normal(0, 1, 1000)
s2 = s1 + offset + noise
s2.name = 's2'

## hedge ratio
lr = LinearRegression()
lr.fit(s1.values.reshape(-1,1),s2.values.reshape(-1,1))
hedge_ratio = lr.coef_[0][0]

#spread
spread = s2 - s1 * hedge_ratio<jupyter_output><empty_output><jupyter_text>Question Do you think we'll need the intercept when calculating the spread?  Why or why not? Since the intercept is a constant, it's not necesary to include it in the spread, since it just shifts the spread up by a constant.  We use the spread to check when it deviates from its historical average, so what matters going foward is how the spread differs from this average.  Quiz Check if spread is stationary using Augmented Dickey Fuller TestThe [adfuller](http://www.statsmodels.org/dev/generated/statsmodels.tsa.stattools.adfuller.html) function is part of the statsmodel library.```adfuller(x, maxlag=None, regression='c', autolag='AIC', store=False, regresults=False)[source]adf (float) – Test statisticpvalue (float) – p-value...```<jupyter_code>def is_spread_stationary(spread, p_level=0.05):
    """
    spread: obtained from linear combination of two series with a hedge ratio
    
    p_level: level of significance required to reject null hypothesis of non-stationarity
    
    returns:
        True if spread can be considered stationary
        False otherwise
    """
    adf_result = adfuller(spread)
    pvalue = adf_result[1]
    print(f"pvalue {pvalue:.4f}")
    if pvalue <= p_level:
        print(f"pvalue is <= {p_level}, assume spread is stationary")
        return True
    else:
        print(f"pvalue is > {p_level}, assume spread is not stationary")
        return False
    
quiz_tests.test_is_spread_stationary(is_spread_stationary)
# Try out your function
print(f"Are the two series candidates for pairs trading? {is_spread_stationary(spread)}")<jupyter_output>pvalue 0.0000
pvalue is <= 0.05, assume spread is stationary
Are the two series candidates for pairs trading? True
{'fraction_non_alphanumeric': 0.0908740854832499, 'fraction_numerical': 0.02117828263380824, 'mean_word_length': 4.515923566878981, 'pattern_counts': {'":': 0, '<': 11, '<?xml version=': 0, '>': 9, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '312157', 'license': 'Apache-2.0', 'n_tokens_mistral': 869, 'n_tokens_neox': 802, 'n_words': 284, 'path': 'Quiz/m2_advanced_quants/l6_pairs_trading_and_mean_reversion/pairs_candidates_solution.ipynb', 'repo_name': 'scumabo/AI4Trading', 'size': '2597'}