<jupyter_start><jupyter_text>Midterm 1AStudent: <NAME>  1. Short Answer 1. FALSE. Mean-variance optimization attempts to optimize the sharpe ratio (mean/variance ratio) of a collection of assets in a portfolio. MV can suggest taking a long or short position in some assets that have a higer or lower sharpe ratio, to the extent that they reduce volatility.  2. FALSE. LETF may make more sense for a long-term or short-term horizon, depending on your goals. Notably, when contrasted with a straight leveraged position, LETF tends to reset leverage rates daily: when comparing the compounded returns between LETF and straight leverage, LETF can out perform or underperform, depending on the level of daily resets as compared to straight-leverage. 3. We should probably include an intercept in this case, given the lack of historically available data to ballpark mean return. Typically, we go w/o an intercept to mimic the full return of a port. we are trying to replicate. However, given our lack of confidence in the mean, it's probably best to simply model with an intercept (replicating vol only) and then update down the road (once we are confident in  a mean estimate.) 4. HDG was pretty effective in tracking HFRI both in and out of sample. From our prior homework, it was apparent that HDG tracked ML and HFRI genereally >90% in correlation. 5. It can happen for a number of reasons, not the least of which is an unusual out-perfomance of the market/benchmark in our sample set (i.e., SPY's relative strength in our case given the strong perfomance of SPY in our sample set). Its not to suggest that Hedge Funds can't provide a positive alpha over time, our negative alpha estimate may simply be a function of really strong market performance on the benchmark.  2. Allocation<jupyter_code>import pandas as pd
import numpy as np
import statsmodels.api as sm
import sklearn as sk
import seaborn as sns
import scipy.stats
import warnings
warnings.filterwarnings('ignore')
# Import DataFrames needed for problem set
df_descriptions =pd.read_excel("C:/Users/setha/Downloads/proshares_analysis_data.xlsx", sheet_name='descriptions', index_col=0).dropna()
df_hedge =pd.read_excel("C:/Users/setha/Downloads/proshares_analysis_data.xlsx", sheet_name='hedge_fund_series', index_col=0).dropna()
df_merrill =pd.read_excel("C:/Users/setha/Downloads/proshares_analysis_data.xlsx", sheet_name='merrill_factors', index_col=0).dropna()
df_merrill_e = df_merrill.subtract(df_merrill['USGG3M Index'], axis=0)
df_merrill_e = df_merrill_e.drop(columns=['USGG3M Index'])
def compute_tangency(df_tilde, diagonalize_Sigma=False):

    """Compute tangency portfolio given a set of excess returns.   
    Also, for convenience, this returns the associated vector of average
    returns and the variance-covariance matrix.
    
    Parameters
    ----------
    diagonalize_Sigma: bool
        When `True`, set the off diagonal elements of the variance-covariance
        matrix to zero.
    """
    Sigma = df_tilde.cov()
    
    # N is the number of assets
    N = Sigma.shape[0]
    Sigma_adj = Sigma.copy()
    if diagonalize_Sigma:
        Sigma_adj.loc[:,:] = np.diag(np.diag(Sigma_adj))
    mu_tilde = df_tilde.mean()
    Sigma_inv = np.linalg.inv(Sigma_adj)
    weights = Sigma_inv @ mu_tilde / (np.ones(N) @ Sigma_inv @ mu_tilde)
    # For convenience, I'll wrap the solution back into a pandas.Series object.
    omega_tangency = pd.Series(weights, index=mu_tilde.index)
    return omega_tangency, mu_tilde, Sigma_adj
def target_mv_portfolio(df_tilde, target_return=0.01, diagonalize_Sigma=False):

    """Compute MV optimal portfolio, given target return and set of excess returns.
    Parameters
    ----------

    diagonalize_Sigma: bool
        When `True`, set the off diagonal elements of the variance-covariance
        matrix to zero.
    """

    omega_tangency, mu_tilde, Sigma = compute_tangency(df_tilde, diagonalize_Sigma=diagonalize_Sigma)
    Sigma_adj = Sigma.copy()

    if diagonalize_Sigma:
        Sigma_adj.loc[:,:] = np.diag(np.diag(Sigma_adj))
    Sigma_inv = np.linalg.inv(Sigma_adj)
    N = Sigma_adj.shape[0]
    delta_tilde = ((np.ones(N) @ Sigma_inv @ mu_tilde)/(mu_tilde @ Sigma_inv @ mu_tilde)) * target_return
    omega_star = delta_tilde * omega_tangency
    return omega_star, mu_tilde, Sigma_adj
def portfolio_stats(omega, mu_tilde, Sigma, annualize_fac):
    # Mean
    mean = (mu_tilde @ omega) * annualize_fac

    # Volatility
    vol = np.sqrt(omega @ Sigma @ omega) * np.sqrt(annualize_fac)

    # Sharpe ratio
    sharpe_ratio = mean / vol

    return round(pd.DataFrame(data = [mean, vol, sharpe_ratio], 
                              index = ['Mean', 'Volatility', 'Sharpe'], 
                              columns = ['Portfolio Stats']), 4)<jupyter_output><empty_output><jupyter_text>1. Weights of $w^{tan}$:<jupyter_code>w_tan = compute_tangency(df_merrill_e)[0]
w_tan<jupyter_output><empty_output><jupyter_text>2.  Weights of $w^*$ with target $\mu = .02$:<jupyter_code>w_star=target_mv_portfolio(df_merrill_e, target_return=0.02)[0]
w_star
sum(w_star)<jupyter_output><empty_output><jupyter_text>Yes, the optimal portfolio, $w*$ is invested in the risk-free asset (it would be short the risk-free about 15-16% of the portfolio given the sum outweighs 100% in the proportions solved above). 3. Mean, Vol and Sharpe Ratio of $w*$ are tabulated below.<jupyter_code>portfolio_stats(target_mv_portfolio(df_merrill_e, target_return=0.01)[0], target_mv_portfolio(df_merrill_e, target_return=0.01)[1], target_mv_portfolio(df_merrill_e, target_return=0.01)[2], 12)<jupyter_output><empty_output><jupyter_text>4.  Weights of $w^*$ with target $\mu = .02$ (using data through 2018), calculations of performance using these weights from 2019-21.<jupyter_code>w_star=target_mv_portfolio(df_merrill_e.loc[:'2018'], target_return=0.02)
w_star[0]
mean_vector_2019_21 = w_star=target_mv_portfolio(df_merrill_e.loc['2019':], target_return=0.02)[1]
mean_vector_2019_21 
portfolio_stats(w_star[0], mean_vector_2019_21, w_star[2], 12)<jupyter_output><empty_output><jupyter_text>5. I would venture to guess that the OOS problem would be worse for these  commodity futures, as there may be substantial correlation between these factors. Commodity futures are likely to trade much more in sync (i.e., pose a correlation across factors), that would likely not help much in making M-for-V balancing trades to optimize variance. In our equity modelling examples, we had largely had a set of variables that were at least somewhat dispersed in their inter-factor correlation and co-variance.  3 Hedging & Replication 1. Optimal hedge ratios = beta out of the OLS run below.<jupyter_code>def univariate_OLS(df, list_of_Y_Cols, X):
    summary = pd.DataFrame(columns=['Alpha', 'Beta', 'Ann_Treynor_Ratio', 'Ann_Info_Ratio'], index=df.columns)
    for curr_factor in list_of_Y_Cols:
        Y = df.iloc[:,curr_factor]
        x = sm.add_constant(X)
        model = sm.OLS(Y, x, missing='drop').fit()
        summary.Alpha[curr_factor] = model.params[0]
        summary.Beta[curr_factor] = model.params[1]
        summary.Ann_Treynor_Ratio[curr_factor] = (df_hedge.iloc[:,curr_factor].mean()*12)/summary.Beta[curr_factor]
        summary.Ann_Info_Ratio[curr_factor] = summary.Alpha[curr_factor]/(np.sqrt(np.sum(model.resid**2)/model.df_resid))*np.sqrt(12)
    return summary
df_merrill_e.columns
X=df_merrill_e.iloc[:, 0]
Y=df_merrill_e.iloc[:,1]
model = sm.OLS(Y, X, missing='drop').fit()
model.summary()<jupyter_output><empty_output><jupyter_text>For every dollar in EEM, we would want to invest -0.9257 (beta above) in SPY to hedge our long EEM position against the movement in SPY. 2. Mean, Vol and SR for hedged position:<jupyter_code>beta=model.params[0]
hedged_pos = -beta*df_merrill_e.iloc[:, 0] +df_merrill_e.iloc[:,1]

mean=hedged_pos.mean()*12
vol=hedged_pos.std()*np.sqrt(12)
SR = mean/vol

print("Mean of Hedged Position: ", mean)
print("Vol of Hedged Position: ", vol)
print("SR of Hedged Position: ", SR)
print("Mean of EEM:")
df_merrill_e.iloc[:,1].mean()*12<jupyter_output>Mean of EEM:<jupyter_text>3. Naturally, the hedged position will not have the same mean as EEM, due to the SPY hedge. Unfortunately, given the strong performance of SPY (generally) and in particular, relative to EEM, the hedged position ended up with a slightly negative mean return vis-a-vis the 3.78% or so EEM could have earned on its own, unhedged.<jupyter_code>df_merrill_e.iloc[:, 0].corr(df_merrill_e.iloc[:, 4])<jupyter_output><empty_output><jupyter_text>4. Given the high correlation(88.16% per above) between SPY and IWN, including IWN may be difficult for hedging, and the OLS may not provide true statistically significant betas. Certainly not something we would use to actually hedge if we were indeed not comfortable/confident in our beta (i.e. our hedge ratio).  4.  Modeling Risk 1. How confident are we that SPY will overperform EFA over the next 10yrs:<jupyter_code>means=df_merrill.mean()*12
vols=df_merrill.mean()*np.sqrt(12)
df_logs=np.log(df_merrill+1)
z = -np.sqrt(10)*(means[2]-means[0])/vols[0]
prob = scipy.stats.norm.cdf(z)
print("Probability that SPY will outperform EEM:")
prob<jupyter_output>Probability that SPY will outperform EEM:<jupyter_text>As calculated above, we can be 99.9% confident based on data from the sample set that SPY will outperform EEM over the next 10yrs. 2. Finding the 1-month 1% VaR<jupyter_code>vol_rolling = df_logs['EFA US Equity'].shift(1).dropna().rolling(60).apply(lambda x: ((x**2).sum()/len(x))**(0.5))
vol_latest =vol_rolling[-1]

# Let us use -2.33 as our z-score for 1% VaR, mu is 0, 
#so it's just z score*vol
prob = -2.33*vol_latest
print("1-month EFA VaR:")
prob<jupyter_output>1-month EFA VaR:
{'fraction_non_alphanumeric': 0.10056381342901076, 'fraction_numerical': 0.01865709892362891, 'mean_word_length': 4.752358490566038, 'pattern_counts': {'":': 0, '<': 39, '<?xml version=': 0, '>': 40, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '353212', 'license': 'MIT', 'n_tokens_mistral': 3391, 'n_tokens_neox': 3148, 'n_words': 1015, 'path': 'solutions/mid1/submissions/sethanirudh_175313_6241691_Midterm #1A.ipynb', 'repo_name': 'tulyu96/finm-portfolio-2021', 'size': '9761'}