<jupyter_start><jupyter_text>Data CleaningIn this section, you will need to read the CSV files into DataFrames and perform any necessary data cleaning steps. After cleaning, combine all DataFrames into a single DataFrame.Files:* `whale_returns.csv`: Contains returns of some famous "whale" investors' portfolios.* `algo_returns.csv`: Contains returns from the in-house trading algorithms from Harold's company.* `sp_tsx_history.csv`: Contains historical closing prices of the S&P TSX 60 Index.  Whale ReturnsRead the Whale Portfolio daily returns and clean the data.<jupyter_code>whale_returns_path  = Path("Resources/whale_returns.csv")
algo_returns_path   = Path("Resources/algo_returns.csv")
sp_tsx_history_path = Path("Resources/sp_tsx_history.csv")

whale_returns_df  = pd.read_csv(whale_returns_path, index_col="Date", parse_dates=True, infer_datetime_format=True)
algo_returns_df   = pd.read_csv(algo_returns_path, index_col="Date", parse_dates=True, infer_datetime_format=True)
sp_tsx_history_df = pd.read_csv(sp_tsx_history_path, index_col='Date', parse_dates=True, infer_datetime_format=True)


# Reading whale returns
whale_returns_df.head()  
# Count nulls

whale_returns_df.isnull().sum()
# Drop nulls

whale_returns_df = whale_returns_df.dropna().copy()
whale_returns_df.isnull().sum()<jupyter_output><empty_output><jupyter_text>Algorithmic Daily ReturnsRead the algorithmic daily returns and clean the data.<jupyter_code># Reading algorithmic returns
algo_returns_df.head()
# Count nulls
algo_returns_df.isnull().sum()
# Drop nulls
algo_returns_df = algo_returns_df.dropna().copy()
algo_returns_df.isnull().sum()<jupyter_output><empty_output><jupyter_text>S&P TSX 60 ReturnsRead the S&P TSX 60 historic closing prices and create a new daily returns DataFrame from the data.<jupyter_code># Reading S&P TSX 60 Closing Prices

sp_tsx_history_df.head()
# Check Data Types
sp_tsx_history_df.dtypes
# Fix Data Types
sp_tsx_history_df["Close"] = sp_tsx_history_df["Close"].str.replace("$", "")
sp_tsx_history_df["Close"] = sp_tsx_history_df["Close"].str.replace(",", "")
sp_tsx_history_df["Close"]
sp_tsx_history_df["Close"] = sp_tsx_history_df["Close"].astype("float")
sp_tsx_history_df.dtypes
# Calculate Daily Returns
sp60_daily_returns = sp_tsx_history_df.pct_change()
sp60_daily_returns.head()
# Drop nulls
sp60_daily_returns = sp60_daily_returns.dropna().copy()
sp60_daily_returns.isnull().sum()
 # Rename `Close` Column to be specific to this portfolio.

sp60_daily_returns = sp60_daily_returns.rename(columns={"Close": "S&P TSX"})
sp60_daily_returns.head()<jupyter_output><empty_output><jupyter_text>Combine Whale, Algorithmic, and S&P TSX 60 Returns<jupyter_code># Join Whale Returns, Algorithmic Returns, and the S&P TSX 60 Returns into a single DataFrame with columns for each portfolio's returns.

daily_returns = pd.concat(
   [whale_returns_df,algo_returns_df, sp60_daily_returns], axis="columns", join="inner"
)
all_portfolios = daily_returns.dropna().copy()<jupyter_output><empty_output><jupyter_text>---  Conduct Quantitative AnalysisIn this section, you will calculate and visualize performance and risk metrics for the portfolios.  Performance Anlysis Calculate and Plot the daily returns.<jupyter_code># Plot daily returns of all portfolios

all_portfolios.plot(figsize=(20,10),title="Daily Returns of All Portfolios", ylabel = "Daily Returns", legend=True)<jupyter_output><empty_output><jupyter_text>Calculate and Plot cumulative returns.<jupyter_code># Calculate cumulative returns of all portfolios

cumulative_returns = (1 + all_portfolios).cumprod() -1



# Plot cumulative returns
cumulative_returns.plot(figsize=(20,10),title="Analysis the Cumulative Returns",xlabel='Date', ylabel='Cumlative Returns')<jupyter_output><empty_output><jupyter_text>---  Risk AnalysisDetermine the _risk_ of each portfolio:1. Create a box plot for each portfolio. 2. Calculate the standard deviation for all portfolios.4. Determine which portfolios are riskier than the S&P TSX 60.5. Calculate the Annualized Standard Deviation.  Create a box plot for each portfolio<jupyter_code># Box plot to visually show risk

all_portfolios.plot.box(figsize=(20,10),title="Portfolio Risk Analysis", xlabel= "Company name", ylabel="Range")<jupyter_output><empty_output><jupyter_text>Calculate Standard Deviations<jupyter_code># Calculate the daily standard deviations of all portfolios
daily_std = all_portfolios.std()
daily_std<jupyter_output><empty_output><jupyter_text>Determine which portfolios are riskier than the S&P TSX 60<jupyter_code># Calculate  the daily standard deviation of S&P TSX 60
sp60_daily_std = all_portfolios['S&P TSX'].std()


# Determine which portfolios are riskier than the S&P TSX 60
daily_std > sp60_daily_std<jupyter_output><empty_output><jupyter_text>Calculate the Annualized Standard Deviation<jupyter_code># Calculate the annualized standard deviation (252 trading days)
annualized_std = daily_std * np.sqrt(252)
annualized_std<jupyter_output><empty_output><jupyter_text>---  Rolling StatisticsRisk changes over time. Analyze the rolling statistics for Risk and Beta. 1. Calculate and plot the rolling standard deviation for the S&P TSX 60 using a 21-day window.2. Calculate the correlation between each stock to determine which portfolios may mimick the S&P TSX 60.3. Choose one portfolio, then calculate and plot the 60-day rolling beta for it and the S&P TSX 60.  Calculate and plot rolling `std` for all portfolios with 21-day window<jupyter_code># Calculate the rolling standard deviation for all portfolios using a 21-day window

all_portfolios_roll_std = all_portfolios.rolling(window=21).std()


# Plot the rolling standard deviation
all_portfolios_roll_std.plot(figsize=(20,10),title=" All Portfolio Rolling Standard Deviation ",  ylabel="Rolling Standard Deviation Range")<jupyter_output><empty_output><jupyter_text>Calculate and plot the correlation<jupyter_code># Calculate the correlation

correlation_df = all_portfolios.corr()


# Display de correlation matrix
correlation_df.style.background_gradient(cmap='gist_rainbow')<jupyter_output><empty_output><jupyter_text>Calculate and Plot Beta for a chosen portfolio and the S&P 60 TSX<jupyter_code># Calculate covariance of a single portfolio
covariance = all_portfolios['SOROS FUND MANAGEMENT LLC'].rolling(60).cov(all_portfolios['S&P TSX'])
#print(covariance)


# Calculate variance of S&P TSX
variance = all_portfolios['S&P TSX'].rolling(60).var()
#print(variance)


# Computing beta
beta = covariance / variance
#print(beta)


# Plot beta trend
beta.plot(figsize=(20,10),title = 'SOROS FUND MANAGEMENT LLC Beta', ylabel = 'Range')<jupyter_output><empty_output><jupyter_text>Rolling Statistics Challenge: Exponentially Weighted Average An alternative way to calculate a rolling window is to take the exponentially weighted moving average. This is like a moving window average, but it assigns greater importance to more recent observations. Try calculating the [`ewm`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.ewm.html) with a 21-day half-life.<jupyter_code># Use `ewm` to calculate the rolling window

all_portfolios.ewm(21).std().plot(figsize=(20,10),title = 'Exponentially Weighted Average', ylabel = 'Range')<jupyter_output><empty_output><jupyter_text>---  Sharpe RatiosIn reality, investment managers and thier institutional investors look at the ratio of return-to-risk, and not just returns alone. After all, if you could invest in one of two portfolios, and each offered the same 10% return, yet one offered lower risk, you'd take that one, right? Using the daily returns, calculate and visualize the Sharpe ratios using a bar plot<jupyter_code># Annualized Sharpe Ratios                        
sharpe_ratios = (all_portfolios.mean() * 252) / (daily_std * np.sqrt(252))
sharpe_ratios.copy()
# Visualize the sharpe ratios as a bar plot
sharpe_ratios.plot.bar(title="Sharpe Ratios")<jupyter_output><empty_output><jupyter_text>Determine whether the algorithmic strategies outperform both the market (S&P TSX 60) and the whales portfolios.Write your answer here!Algo 1 outperforms both the market (S&P TSX 60) and the whales' portfolios.  The market (S&P TSX 60) Annualized Sharpe Ratio is 0.19, and the whales' portfolios Annualized Sharpe Ratio is as: Soros Fund Management Llc: 0.286709; Paulson & Co.Inc: -0.547594; Tiger Global Management Llc: -0.144455; Berkshire Hathaway Inc: 0.467045)Algo 1 Annualized Sharpe Ratios is 1.49, more significant than 1.0. It not only outperforms other portfolios, but Algo 1 is also deemed acceptable to good for investors in terms of a high degree of expected return for a relatively low amount of risk.Algo 2 Annualized Sharpe Ratios is 0.39, and it outperforms the market (S&P TSX 60)  (Sharpe Ratio 0.19) and three stocks in whales' portfolios (Sharpe Ratio: Soros Fund Management Llc:  0.286709; Paulson & Co.Inc: -0.547594; Tiger Global Management Llc:  -0.144455). However, because Algo 2 Annualized Sharpe Ratios is below 1.0 is considered sub-optimal for investors.  --------------------  Create Custom PortfolioIn this section, you will build your own portfolio of stocks, calculate the returns, and compare the results to the Whale Portfolios and the S&P TSX 60. 1. Choose 3-5 custom stocks with at last 1 year's worth of historic prices and create a DataFrame of the closing prices and dates for each stock.2. Calculate the weighted returns for the portfolio assuming an equal number of shares for each stock.3. Join your portfolio returns to the DataFrame that contains all of the portfolio returns.4. Re-run the performance and risk analysis with your portfolio to see how it compares to the others.5. Include correlation analysis to determine which stocks (if any) are correlated.  Choose 3-5 custom stocks with at last 1 year's worth of historic prices and create a DataFrame of the closing prices and dates for each stock.For this demo solution, we fetch data from three companies listes in the S&P TSX 60 index.* `SHOP` - [Shopify Inc](https://en.wikipedia.org/wiki/Shopify)* `OTEX` - [Open Text Corporation](https://en.wikipedia.org/wiki/OpenText)* `L` - [Loblaw Companies Limited](https://en.wikipedia.org/wiki/Loblaw_Companies)<jupyter_code># Reading data from 1st stock
amzn_path = Path("Resources/amzn.csv")
amzn_df = pd.read_csv(amzn_path, index_col='Date', parse_dates=True, infer_datetime_format=True)
amzn_df.head()
# Reading data from 2nd stock
axp_path = Path("Resources/axp.csv")
axp_df = pd.read_csv(axp_path, index_col='Date', parse_dates=True, infer_datetime_format=True)
axp_df.head()
# Reading data from 3rd stock
delta_path = Path("Resources/delta.csv")
delta_df = pd.read_csv(delta_path, index_col='Date', parse_dates=True, infer_datetime_format=True)
delta_df.head()
# Combine all stocks in a single DataFrame
all_stocks_group2 = pd.concat([amzn_df,axp_df,delta_df], axis="columns", join="inner")
all_stocks_group2.head().dropna().copy()
# Reset Date index
all_stocks_group2.head()    # no need to reset date index since Date originally was in index    
# Reorganize portfolio data by having a column per symbol
columns = ["AMZN", "AXP", "Delta"]
all_stocks_group2.columns = columns
all_stocks_group2.sort_index().head()
# Calculate daily returns
all_stocks_group2_daily_returns = all_stocks_group2.pct_change()

# Drop NAs
all_stocks_group2_daily_returns = all_stocks_group2_daily_returns.dropna()

# Display sample data
all_stocks_group2_daily_returns.plot(figsize=(20,10),title = 'AMZN, AXP, Delta Daily Returns', ylabel = 'Range')<jupyter_output><empty_output><jupyter_text>Calculate the weighted returns for the portfolio assuming an equal number of shares for each stock<jupyter_code># Set weights
weights = [1/3, 1/3, 1/3]

# Calculate portfolio return
group2_portfolio_returns = all_stocks_group2_daily_returns.dot(weights)
group2_portfolio_returns.tail()


# Display sample data
group2_portfolio_returns.plot(figsize=(20,10),title = 'Portfolio Return', ylabel = 'Range')<jupyter_output><empty_output><jupyter_text>Join your portfolio returns to the DataFrame that contains all of the portfolio returns<jupyter_code># Join your returns DataFrame to the original returns DataFrame
all_portfolios_returns= all_portfolios
all_portfolios_returns["Stocks_Group2"] = group2_portfolio_returns
all_portfolios_returns.tail()
# Only compare dates where return data exists for all the stocks (drop NaNs)
all_portfolios_returns.dropna(inplace=True)
all_portfolios_returns.head()<jupyter_output><empty_output><jupyter_text>Re-run the risk analysis with your portfolio to see how it compares to the others  Calculate the Annualized Standard Deviation<jupyter_code># Calculate the annualized `std`

all_portfolios_daily_std = all_portfolios_returns.std()  #(have to calculate Daily Standard Deviations) 
all_portfolios_daily_std.head()


all_portfolios_annualized_std = all_portfolios_daily_std * np.sqrt(252)
all_portfolios_annualized_std<jupyter_output><empty_output><jupyter_text>Calculate and plot rolling `std` with 21-day window<jupyter_code># Calculate rolling standard deviation
all_portfolois_rolling = all_portfolios_returns.rolling(window=21).std()

# Plot rolling standard deviation
all_portfolois_rolling.plot(figsize=(20,10),title = 'Portfolio Return', ylabel = 'Range')<jupyter_output><empty_output><jupyter_text>Calculate and plot the correlation<jupyter_code># Calculate and plot the correlation
price_correlation = all_portfolios_returns.corr()
price_correlation
plt.figure(figsize=(20,10))
sns.heatmap(price_correlation, vmin=-1, vmax=1, annot=True)<jupyter_output><empty_output><jupyter_text>Calculate and Plot the 60-day Rolling Beta for Your Portfolio compared to the S&P 60 TSX<jupyter_code># Calculate and plot Beta
covariance = all_portfolios_returns['Stocks_Group2'].rolling(60).cov(all_portfolios['S&P TSX'])
variance = all_portfolios_returns['S&P TSX'].rolling(60).var()
beta = covariance / variance
beta.plot(figsize=(20,10),title = '60 day Rolling Beta of my Portfolio VS S&P 60 TSX', ylabel = 'Range')<jupyter_output><empty_output><jupyter_text>Using the daily returns, calculate and visualize the Sharpe ratios using a bar plot<jupyter_code># Calculate Annualzied Sharpe Ratios
sharpe_ratios = (all_portfolios_returns.mean() * 252) / (all_portfolios_daily_std * np.sqrt(252))
sharpe_ratios.copy()
# Visualize the sharpe ratios as a bar plot
sharpe_ratios.plot.bar(title="Sharpe Ratios")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08992183717230408, 'fraction_numerical': 0.01943695095801342, 'mean_word_length': 6.70682302771855, 'pattern_counts': {'":': 1, '<': 93, '<?xml version=': 0, '>': 94, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '23', 'dir': 'jupyter-structured-clean-dedup', 'id': '297973', 'license': 'MIT', 'n_tokens_mistral': 4901, 'n_tokens_neox': 4512, 'n_words': 1378, 'path': 'sub 4/risk return anylysis/risk return anaylysis sub 1/whale_analysis.ipynb', 'repo_name': 'nwpistols/risk-return-analysis', 'size': '14457'}