<jupyter_start><jupyter_text>A Whale off the Port(folio) --- In this assignment, you'll get to use what you've learned this week to evaluate the performance among various algorithmic, hedge, and mutual fund portfolios and compare them against the S&P TSX 60 Index.<jupyter_code># Initial imports
import pandas as pd
import numpy as np
import datetime as dt
from pathlib import Path

%matplotlib inline<jupyter_output><empty_output><jupyter_text>Data CleaningIn this section, you will need to read the CSV files into DataFrames and perform any necessary data cleaning steps. After cleaning, combine all DataFrames into a single DataFrame.Files:* `whale_returns.csv`: Contains returns of some famous "whale" investors' portfolios.* `algo_returns.csv`: Contains returns from the in-house trading algorithms from Harold's company.* `sp_tsx_history.csv`: Contains historical closing prices of the S&P TSX 60 Index.  Whale ReturnsRead the Whale Portfolio daily returns and clean the data.<jupyter_code>csv_path1 = Path ("Resources/whale_returns.csv")
csv_path2 = Path ("Resources/algo_returns.csv")
csv_path3 = Path ("Resources/sp_tsx_history.csv")

whale_returns_df = pd.read_csv(csv_path1, index_col="Date", infer_datetime_format=True, parse_dates=True)
algo_returns_df = pd.read_csv(csv_path2, index_col="Date", infer_datetime_format=True, parse_dates=True)
sp_tsx_df = pd.read_csv(csv_path3, index_col="Date", infer_datetime_format=True, parse_dates=True)
# Reading whale returns
whale_returns_df.head()
# Count nulls
whale_returns_df.isnull().sum()
# Drop nulls
whale_returns_df = whale_returns_df.dropna()
whale_returns_df.head()<jupyter_output><empty_output><jupyter_text>Algorithmic Daily ReturnsRead the algorithmic daily returns and clean the data.<jupyter_code># Reading algorithmic returns
algo_returns_df.head()
# Count nulls
algo_returns_df.isnull().sum()
# Drop nulls
algo_returns_df = algo_returns_df.dropna()
algo_returns_df.head()<jupyter_output><empty_output><jupyter_text>S&P TSX 60 ReturnsRead the S&P TSX 60 historic closing prices and create a new daily returns DataFrame from the data.<jupyter_code># Reading S&P TSX 60 Closing Prices
sp_tsx_df.head()
# Check Data Types
sp_tsx_df.dtypes
# Fix Data Types
sp_tsx_df['Close'] = sp_tsx_df['Close'].str.replace('$', '')
sp_tsx_df['Close'] = sp_tsx_df['Close'].str.replace(',', '')
sp_tsx_df['Close'] = sp_tsx_df['Close'].astype(float)

sp_tsx_df.dtypes


# Calculate Daily Returns
sp_tsx_daily = sp_tsx_df.pct_change()
sp_tsx_daily.head(20)
# Drop nulls
sp_tsx_daily.isnull().sum()
sp_tsx_daily = sp_tsx_daily.dropna()
sp_tsx_daily.head()
# Rename `Close` Column to be specific to this portfolio.
sp_tsx_daily = sp_tsx_daily.rename({"Close":"SP Daily Returns"}, axis='columns')

sp_tsx_daily.head()<jupyter_output><empty_output><jupyter_text>Combine Whale, Algorithmic, and S&P TSX 60 Returns<jupyter_code># Join Whale Returns, Algorithmic Returns, and the S&P TSX 60 Returns into a single DataFrame with columns for each portfolio's returns.

combined_returns = pd.concat([whale_returns_df, algo_returns_df, sp_tsx_daily], axis="columns", join="inner")
combined_returns.head()<jupyter_output><empty_output><jupyter_text>---  Conduct Quantitative AnalysisIn this section, you will calculate and visualize performance and risk metrics for the portfolios.  Performance Anlysis Calculate and Plot the daily returns.<jupyter_code># Plot daily returns of all portfolios
combined_returns.plot()<jupyter_output><empty_output><jupyter_text>Calculate and Plot cumulative returns.<jupyter_code># Calculate cumulative returns of all portfolios
cum_returns = (1 + combined_returns).cumprod() - 1
# Plot cumulative returns
cum_returns.plot(figsize=(14, 9))<jupyter_output><empty_output><jupyter_text>---  Risk AnalysisDetermine the _risk_ of each portfolio:1. Create a box plot for each portfolio. 2. Calculate the standard deviation for all portfolios.4. Determine which portfolios are riskier than the S&P TSX 60.5. Calculate the Annualized Standard Deviation.  Create a box plot for each portfolio<jupyter_code># Box plot to visually show risk

whales_portfolio = whale_returns_df.plot(kind="box", figsize=(14, 9))


algo_portfolio = algo_returns_df.plot(kind="box",figsize=(14, 9))
sp_portfolio = sp_tsx_daily.plot(kind="box",figsize=(14, 9))<jupyter_output><empty_output><jupyter_text>Calculate Standard Deviations<jupyter_code># Calculate the daily standard deviations of all portfolios
#whale_returns_mean = whale_returns_df.mean()
whale_returns_std = whale_returns_df.std().mean()
#algo_returns_mean = algo_returns_df.mean()
algo_returns_std = algo_returns_df.std().mean()<jupyter_output><empty_output><jupyter_text>Determine which portfolios are riskier than the S&P TSX 60<jupyter_code># Calculate  the daily standard deviation of S&P TSX 60
sp_returns_std = sp_tsx_daily.std()
print(f"Whale portfolio std : {whale_returns_std}")
print(f"Algo portfolio std  : {algo_returns_std}")
print(sp_returns_std)
# Determine which portfolios are riskier than the S&P TSX 60


### Whale portfolio and Algo portfolio are riskier than S&P<jupyter_output><empty_output><jupyter_text>Calculate the Annualized Standard Deviation<jupyter_code># Calculate the annualized standard deviation (252 trading days)
annualized_whale_std = whale_returns_std * np.sqrt(252)
annualized_algo_std = algo_returns_std * np.sqrt(252)

annualized_sp_std = sp_returns_std * np.sqrt(252)

print(annualized_whale_std)
print(annualized_algo_std)
print(annualized_sp_std)<jupyter_output>0.15268850476046356
0.13060096487608586
SP Daily Returns    0.105365
dtype: float64<jupyter_text>---  Rolling StatisticsRisk changes over time. Analyze the rolling statistics for Risk and Beta. 1. Calculate and plot the rolling standard deviation for all portfolios using a 21-day window.2. Calculate the correlation between each stock to determine which portfolios may mimick the S&P TSX 60.3. Choose one portfolio, then calculate and plot the 60-day rolling beta for it and the S&P TSX 60.  Calculate and plot rolling `std` for all portfolios with 21-day window<jupyter_code># Calculate the rolling standard deviation for all portfolios using a 21-day window


whale_returns_df.rolling(window =21).std().plot(figsize=(11, 6))
algo_returns_df.rolling(window =21).std().plot(figsize=(11, 6))
sp_tsx_daily.rolling(window =21).std().plot(figsize=(11, 6))
#Plot the rolling standard deviation<jupyter_output><empty_output><jupyter_text>Calculate and plot the correlation<jupyter_code># Calculate the correlation
correlation = combined_returns.corr()
# Display de correlation matrix

#import seaborn as sns
correlation.style.background_gradient(cmap="winter")<jupyter_output><empty_output><jupyter_text>Calculate and Plot Beta for a chosen portfolio and the S&P 60 TSX<jupyter_code># Calculate covariance of a single portfolio
cov_sp = combined_returns["Algo 1"].rolling(21).mean().cov(combined_returns["SP Daily Returns"])

# Calculate variance of S&P TSX
var_sp_portfolio = combined_returns["SP Daily Returns"].rolling(21).var()



# Computing beta
beta = cov_sp / var_sp_portfolio
# Plot beta trend
beta.plot()<jupyter_output><empty_output><jupyter_text>Rolling Statistics Challenge: Exponentially Weighted Average An alternative way to calculate a rolling window is to take the exponentially weighted moving average. This is like a moving window average, but it assigns greater importance to more recent observations. Try calculating the [`ewm`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.ewm.html) with a 21-day half life for each portfolio, using standard deviation (`std`) as the metric of interest.<jupyter_code># Use `ewm` to calculate the rolling window
combined_returns.ewm(span =21).std().plot()<jupyter_output><empty_output><jupyter_text>---  Sharpe RatiosIn reality, investment managers and thier institutional investors look at the ratio of return-to-risk, and not just returns alone. After all, if you could invest in one of two portfolios, and each offered the same 10% return, yet one offered lower risk, you'd take that one, right? Using the daily returns, calculate and visualize the Sharpe ratios using a bar plot<jupyter_code># Annualized Sharpe Ratios
sharpe_ratio = (combined_returns.mean() *252) / ( combined_returns.std() * np.sqrt(252))
# Visualize the sharpe ratios as a bar plot
sharpe_ratio.plot(kind="bar")
print(sharpe_ratio)<jupyter_output>SOROS FUND MANAGEMENT LLC      0.286709
PAULSON & CO.INC.             -0.547594
TIGER GLOBAL MANAGEMENT LLC   -0.144455
BERKSHIRE HATHAWAY INC         0.467045
Algo 1                         1.491514
Algo 2                         0.396817
SP Daily Returns               0.195550
dtype: float64<jupyter_text>Determine whether the algorithmic strategies outperform both the market (S&P TSX 60) and the whales portfolios.Write your answer here!Algorithmic strategies(expecially Algo 1) outperform S&P and whales portfolio with the highest sharpe ratio of 1.49 compared to 0.46 for whales portfolio and 0.19 for S&P ---  Create Custom PortfolioIn this section, you will build your own portfolio of stocks, calculate the returns, and compare the results to the Whale Portfolios and the S&P TSX 60. 1. Choose 3-5 custom stocks with at last 1 year's worth of historic prices and create a DataFrame of the closing prices and dates for each stock.2. Calculate the weighted returns for the portfolio assuming an equal number of shares for each stock.3. Join your portfolio returns to the DataFrame that contains all of the portfolio returns.4. Re-run the performance and risk analysis with your portfolio to see how it compares to the others.5. Include correlation analysis to determine which stocks (if any) are correlated.  Choose 3-5 custom stocks with at last 1 year's worth of historic prices and create a DataFrame of the closing prices and dates for each stock.<jupyter_code># Reading data from 1st stock
csv_path4 = Path ("Resources/tsla_history.csv")
uber_df = pd.read_csv(csv_path4, index_col="Date", infer_datetime_format=True, parse_dates=True)

# Reading data from 2nd stock
csv_path5 = Path ("Resources/sony_history3.csv")
sony_df = pd.read_csv(csv_path5, index_col="Date", infer_datetime_format=True, parse_dates=True)

# Reading data from 3rd stock
csv_path6 = Path ("Resources/ebay_history3.csv")
ebay_df = pd.read_csv(csv_path6, index_col="Date", infer_datetime_format=True, parse_dates=True)

# Reading data from 4rd stock
csv_path7 = Path ("Resources/ibm_history3.csv")
ibm_df = pd.read_csv(csv_path7, index_col="Date", infer_datetime_format=True, parse_dates=True)

# Combine all stocks in a single DataFrame
my_portfolio_df = pd.concat([uber_df, sony_df, ebay_df, ibm_df], axis="columns", join="inner")
my_portfolio_df.head()
# Reset Date index


# Reorganize portfolio data by having a column per symbol
my_portfolio_df = my_portfolio_df.drop(["Open", "High", "Low", "Volume","Adj Close"], axis=1)
my_portfolio_df = my_portfolio_df.replace('$', '', regex=True)
my_portfolio_df = my_portfolio_df.replace(',', '', regex=True)
my_portfolio_df = my_portfolio_df.astype(float)

my_portfolio_df.head()
                     
# Calculate daily returns

col_name = ["Tsla", "Sony", "Ebay", "Ibm"]
my_portfolio_df.columns = col_name
my_portfolio_dailyr = my_portfolio_df.pct_change()
# Drop NAs
my_portfolio_dailyr.head()
my_portfolio_dailyr.isnull().sum()
my_portfolio_dailyr = my_portfolio_dailyr.dropna()

# Display sample data
my_portfolio_dailyr.sample()<jupyter_output><empty_output><jupyter_text>Calculate the weighted returns for the portfolio assuming an equal number of shares for each stock<jupyter_code># Set weights
weights = [1/4, 1/4, 1/4, 1/4]

# Calculate portfolio return
my_portfolio_returns = my_portfolio_dailyr.dot(weights)
# Display sample data
my_portfolio_returns.head()
my_portfolio_dailyr.head(20)<jupyter_output><empty_output><jupyter_text>Join your portfolio returns to the DataFrame that contains all of the portfolio returns<jupyter_code># Join your returns DataFrame to the original returns DataFrame

joined_returns = pd.concat([my_portfolio_dailyr,combined_returns], axis = "columns", join = "inner")

# Only compare dates where return data exists for all the stocks (drop NaNs)
joined_returns = joined_returns.dropna()
joined_returns.tail()<jupyter_output><empty_output><jupyter_text>Re-run the risk analysis with your portfolio to see how it compares to the others  Calculate the Annualized Standard Deviation<jupyter_code># Calculate the annualized `std`
my_portfolio_dailyr.std()<jupyter_output><empty_output><jupyter_text>Calculate and plot rolling `std` with 21-day window<jupyter_code># Calculate rolling standard deviation
my_port_roll_std = my_portfolio_dailyr.rolling(21).std()
# Plot rolling standard deviation
my_port_roll_std.plot()<jupyter_output><empty_output><jupyter_text>Calculate and plot the correlation<jupyter_code># Calculate and plot the correlation
my_port_corr = joined_returns.corr()
my_port_corr.style.background_gradient(cmap="winter")<jupyter_output><empty_output><jupyter_text>Calculate and Plot the 60-day Rolling Beta for Your Portfolio compared to the S&P 60 TSX<jupyter_code># Calculate and plot Beta
cov_my_port = joined_returns["Ibm"].rolling(21).mean().cov(joined_returns["SP Daily Returns"])

# Calculate variance of S&P TSX
var_my_port = joined_returns["SP Daily Returns"].rolling(21).var()



# Computing beta
beta2 = cov_my_port / var_my_port
# Plot beta trend
beta2.plot()<jupyter_output><empty_output><jupyter_text>Using the daily returns, calculate and visualize the Sharpe ratios using a bar plot<jupyter_code># Calculate Annualized Sharpe Ratios
my_port_sharpe = (joined_returns.mean() *252) / ( joined_returns.std() * np.sqrt(252))
# Visualize the sharpe ratios as a bar plot
my_port_sharpe.plot(kind="bar")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08862223507300852, 'fraction_numerical': 0.02009541708833309, 'mean_word_length': 6.430182599355532, 'pattern_counts': {'":': 1, '<': 95, '<?xml version=': 0, '>': 95, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '24', 'dir': 'jupyter-structured-clean-dedup', 'id': '333394', 'license': 'ADSL', 'n_tokens_mistral': 4707, 'n_tokens_neox': 4363, 'n_words': 1332, 'path': 'Starter_Code/whale_analysis.ipynb', 'repo_name': 'Tanobuon/pandas_homework', 'size': '13834'}