<jupyter_start><jupyter_text>Historical VarianceLet's see how we'd be calculating a covariance matrix of assets without the help of a factor model<jupyter_code>import sys
!{sys.executable} -m pip install -r requirements.txt
import numpy as np
import pandas as pd
import time
import os
import quiz_helper
import matplotlib.pyplot as plt
%matplotlib inline
plt.style.use('ggplot')
plt.rcParams['figure.figsize'] = (14, 8)<jupyter_output><empty_output><jupyter_text>data bundle<jupyter_code>import os
import quiz_helper
from zipline.data import bundles
os.environ['ZIPLINE_ROOT'] = os.path.join(os.getcwd(), '..', '..','data','module_4_quizzes_eod')
ingest_func = bundles.csvdir.csvdir_equities(['daily'], quiz_helper.EOD_BUNDLE_NAME)
bundles.register(quiz_helper.EOD_BUNDLE_NAME, ingest_func)
print('Data Registered')<jupyter_output>Data Registered<jupyter_text>Build pipeline engine<jupyter_code>from zipline.pipeline import Pipeline
from zipline.pipeline.factors import AverageDollarVolume
from zipline.utils.calendars import get_calendar

universe = AverageDollarVolume(window_length=120).top(500) 
trading_calendar = get_calendar('NYSE') 
bundle_data = bundles.load(quiz_helper.EOD_BUNDLE_NAME)
engine = quiz_helper.build_pipeline_engine(bundle_data, trading_calendar)<jupyter_output><empty_output><jupyter_text>View Data¶With the pipeline engine built, let's get the stocks at the end of the period in the universe we're using. We'll use these tickers to generate the returns data for the our risk model.<jupyter_code>universe_end_date = pd.Timestamp('2016-01-05', tz='UTC')

universe_tickers = engine\
    .run_pipeline(
        Pipeline(screen=universe),
        universe_end_date,
        universe_end_date)\
    .index.get_level_values(1)\
    .values.tolist()
    
universe_tickers
len(universe_tickers)
from zipline.data.data_portal import DataPortal

data_portal = DataPortal(
    bundle_data.asset_finder,
    trading_calendar=trading_calendar,
    first_trading_day=bundle_data.equity_daily_bar_reader.first_trading_day,
    equity_minute_reader=None,
    equity_daily_reader=bundle_data.equity_daily_bar_reader,
    adjustment_reader=bundle_data.adjustment_reader)<jupyter_output><empty_output><jupyter_text>Get pricing data helper function<jupyter_code>from quiz_helper import get_pricing<jupyter_output><empty_output><jupyter_text>get pricing data into a dataframe<jupyter_code>returns_df = \
    get_pricing(
        data_portal,
        trading_calendar,
        universe_tickers,
        universe_end_date - pd.DateOffset(years=5),
        universe_end_date)\
    .pct_change()[1:].fillna(0) #convert prices into returns

returns_df<jupyter_output><empty_output><jupyter_text>Quiz 1Check out the [numpy.cov documentation](https://docs.scipy.org/doc/numpy-1.15.0/reference/generated/numpy.cov.html).  Then think about what's wrong with the following use of numpy.cov<jupyter_code># What's wrong with this?
annualization_factor = 252
covariance_assets_not_correct = annualization_factor*np.cov(returns_df)
## TODO: Check the shape of the covariance matrix<jupyter_output><empty_output><jupyter_text>Answer 1 here:  Quiz 2How can you adjust the input so that we get the desired covariance matrix of assets?<jupyter_code># TODO: calculate the covariance matrix of assets
annualization_factor = # ...
covariance_assets = # ...
covariance_assets.shape<jupyter_output><empty_output><jupyter_text>Answer 2:  Visualize the covariance matrix<jupyter_code>import seaborn as sns
# view a heatmap of the covariance matrix
sns.heatmap(covariance_assets,cmap='Paired');
## If the colors aren't distinctive, please try a couple of these color schemes:
## cmap = 'tab10'
# cmap = 'Accent'<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09007797795106211, 'fraction_numerical': 0.009411132024737832, 'mean_word_length': 6.395626242544732, 'pattern_counts': {'":': 0, '<': 36, '<?xml version=': 0, '>': 36, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '9413', 'license': 'MIT', 'n_tokens_mistral': 1293, 'n_tokens_neox': 1232, 'n_words': 268, 'path': 'Risk Factor Models/historical_variance.ipynb', 'repo_name': '2series/Trading-AI-Style', 'size': '3719'}