<jupyter_start><jupyter_text>Trading with Momentum Project AboutIn this project, momentum trading strategies were implement, and tested to see if momentum strategies have the potential to be profitable; trading signal based on a momentum indicator were generated. The data consisted of a universe of stocks and time range (end of day from Quotemedia). The signals were computed for the time range given and applied to the dataset to produce projected returns. Finally, statistical test were performed on the mean of the returns to conclude if there is alpha in the signal. PackagesImport only [Pandas](https://pandas.pydata.org/) and [Numpy](http://www.numpy.org/); the rest of the packages required are in the requirements.txt file .The other packages that were imported are `helper`, `project_helper`, and `project_tests`. These are custom packages built to help you solve the problems.  The `helper` and `project_helper` modules contain utility functions and graph functions. The `project_tests` contains the unit tests for all the problems. Install Packages import sys!{sys.executable} -m pip install -r requirements.txt  Load Packages<jupyter_code>import pandas as pd
import numpy as np
import helper
import project_helper
import project_tests<jupyter_output><empty_output><jupyter_text>Market Data Load DataThe data used end of day data. This contains data for many stocks, but we'll be looking at stocks in the S&P 500. We also made things a little easier to run by narrowing down our range of time period instead of using all of the data.<jupyter_code>df = pd.read_csv('./eod-quotemedia.csv', parse_dates=['date'], index_col=False)

close = df.reset_index().pivot(index='date', columns='ticker', values='adj_close')

print('Data Loaded')<jupyter_output>Data Loaded<jupyter_text>View DataSee how the data looks like for `close`.<jupyter_code>project_helper.print_dataframe(close)<jupyter_output><empty_output><jupyter_text>Stock ExampleLet's see what a single stock looks like from the closing prices. For this example and future display examples in this project, we'll use Netflix's stock (NFLX). If we tried to graph all the stocks, it would be too much information.<jupyter_code>apple_ticker = 'AAPL'
project_helper.plot_stock(close[apple_ticker], '{} Stock'.format(apple_ticker))<jupyter_output><empty_output><jupyter_text>Resample Adjusted PricesThe trading signal developed in this project does not need to be based on daily prices, for instance, it can be month-end prices to perform trading once a month. To do this, we must first resample the daily adjusted closing prices into monthly buckets, and select the last observation of each month.Implement the `resample_prices` to resample `close_prices` at the sampling frequency of `freq`.<jupyter_code>def resample_prices(close_prices, freq='M'):
    """
    Resample close prices for each ticker at specified frequency.
    
    Parameters
    ----------
    close_prices : DataFrame
        Close prices for each ticker and date
    freq : str
        What frequency to sample at
        For valid freq choices, see http://pandas.pydata.org/pandas-docs/stable/timeseries.html#offset-aliases
    
    Returns
    -------
    prices_resampled : DataFrame
        Resampled prices for each ticker and date
    """
    resampled = close_prices.resample(freq).last()
    print(resampled)
    return resampled

project_tests.test_resample_prices(resample_prices)<jupyter_output>UJX         NJK          NFQB        XVEU         FJQ
2008-08-31  21.05081048 17.01384381   10.98450376 11.24809343 12.96171273
2008-09-30 482.34539247 35.20258059 3516.54167823 66.40531433 13.50396048
2008-10-31  10.91893302 17.90864387   24.80126542 12.48895419 10.52435923
2008-11-30  11.54549538 23.98146843   24.97476306 36.03196210 14.30433232
Tests Passed<jupyter_text>View DataLet's apply this function to `close` and view the results.<jupyter_code>monthly_close = resample_prices(close)
project_helper.plot_resampled_prices(
    monthly_close.loc[:, apple_ticker],
    close.loc[:, apple_ticker],
    '{} Stock - Close Vs Monthly Close'.format(apple_ticker))<jupyter_output>ticker               A         AAL          AAP         AAPL        ABBV  \
date                                                                       
2013-07-31 30.77861719 18.63139292  81.73270857  58.73000866 38.52144972   
2013-08-31 32.09288410 15.55986096  79.33492514  63.64994327 36.09056668   
2013-09-30 35.34697923 18.25587647  81.98212977  62.28266407 37.88620154   
2013-10-31 35.00902763 21.15409315  98.34285959  68.28583759 41.39637606   
2013-11-30 36.94707663 22.60801580 100.15741326  73.07037475 41.39637606   
2013-12-31 39.53485221 24.31228275 109.80605374  73.72082947 45.12162270   
2014-01-31 40.19849023 32.30404302 113.90344263  65.78133976 42.40046983   
2014-02-28 39.35511692 35.55851889 126.35434591  69.56208595 43.84740848   
2014-03-31 38.65691442 35.24077420 125.56160936  70.95004943 44.26943225   
2014-04-30 37.44602782 33.76759430 120.39025770  78.00222579 45.26058952   
2014-05-31 39.45552968 38.66856535 123.24889355  84.14255587 47.21597213   
2014-06-30 3[...]<jupyter_text>Compute Log ReturnsCompute log returns ($R_t$) from prices ($P_t$) as your primary momentum indicator:$$R_t = log_e(P_t) - log_e(P_{t-1})$$Implementing the `compute_log_returns` function below, such that it accepts a dataframe (like one returned by `resample_prices`), and produces a similar dataframe of log returns. Use Numpy's [log function](https://docs.scipy.org/doc/numpy/reference/generated/numpy.log.html) to help you calculate the log returns.<jupyter_code>def compute_log_returns(prices):
    """
    Compute log returns for each ticker.
    
    Parameters
    ----------
    prices : DataFrame
        Prices for each ticker and date
    
    Returns
    -------
    log_returns : DataFrame
        Log returns for each ticker and date
    """
    log_returns = np.log(prices) - np.log(prices.shift(1))
    print(log_returns)
    return log_returns

project_tests.test_compute_log_returns(compute_log_returns)<jupyter_output>ZAN         BLA        SGAI         SER         IIU
2008-08-31         nan         nan         nan         nan         nan
2008-09-30  3.13172138  0.72709204  5.76874778  1.77557845  0.04098317
2008-10-31 -3.78816218 -0.67583590 -4.95433863 -1.67093250 -0.24929051
2008-11-30  0.05579709  0.29199789  0.00697116  1.05956179  0.30686995
Tests Passed<jupyter_text>View DataUsing the same data returned from `resample_prices`, we'll generate the log returns.<jupyter_code>monthly_close_returns = compute_log_returns(monthly_close)
project_helper.plot_returns(
    monthly_close_returns.loc[:, apple_ticker],
    'Log Returns of {} Stock (Monthly)'.format(apple_ticker))<jupyter_output>ticker               A         AAL         AAP        AAPL        ABBV  \
date                                                                     
2013-07-31         nan         nan         nan         nan         nan   
2013-08-31  0.04181412 -0.18015337 -0.02977582  0.08044762 -0.06518370   
2013-09-30  0.09657861  0.15979244  0.03282284 -0.02171531  0.04855545   
2013-10-31 -0.00960698  0.14734639  0.18195865  0.09201927  0.08860637   
2013-11-30  0.05388057  0.06647111  0.01828314  0.06772063  0.00000000   
2013-12-31  0.06769609  0.07267716  0.09197258  0.00886237  0.08616823   
2014-01-31  0.01664682  0.28421071  0.03663543 -0.11394918 -0.06220213   
2014-02-28 -0.02120344  0.09598737  0.10373913  0.05588347  0.03355617   
2014-03-31 -0.01790035 -0.00897599 -0.00629368  0.01975642  0.00957880   
2014-04-30 -0.03182502 -0.04270218 -0.04205794  0.09476126  0.02214224   
2014-05-31  0.05227357  0.13552541  0.02346722  0.07577509  0.04229556   
2014-06-30  0.01103586  0.06739797  0.[...]<jupyter_text>Shift ReturnsImplement the `shift_returns` function to shift the log returns to the previous or future returns in the time series. For example, the parameter `shift_n` is 2 and `returns` is the following:```                           Returns               A         B         C         D2013-07-08     0.015     0.082     0.096     0.020     ...2013-07-09     0.037     0.095     0.027     0.063     ...2013-07-10     0.094     0.001     0.093     0.019     ...2013-07-11     0.092     0.057     0.069     0.087     ......            ...       ...       ...       ...```the output of the `shift_returns` function would be:```                        Shift Returns               A         B         C         D2013-07-08     NaN       NaN       NaN       NaN       ...2013-07-09     NaN       NaN       NaN       NaN       ...2013-07-10     0.015     0.082     0.096     0.020     ...2013-07-11     0.037     0.095     0.027     0.063     ......            ...       ...       ...       ...```Using the same `returns` data as above, the `shift_returns` function should generate the following with `shift_n` as -2:```                        Shift Returns               A         B         C         D2013-07-08     0.094     0.001     0.093     0.019     ...2013-07-09     0.092     0.057     0.069     0.087     ......            ...       ...       ...       ...       ......            ...       ...       ...       ...       ......            NaN       NaN       NaN       NaN       ......            NaN       NaN       NaN       NaN       ...```<jupyter_code>def shift_returns(returns, shift_n):
    """
    Generate shifted returns
    
    Parameters
    ----------
    returns : DataFrame
        Returns for each ticker and date
    shift_n : int
        Number of periods to move, can be positive or negative
    
    Returns
    -------
    shifted_returns : DataFrame
        Shifted returns for each ticker and date
    """
    shifted_returns = returns.shift(shift_n)
    print(returns)
    print(shifted_returns)
    return shifted_returns

project_tests.test_shift_returns(shift_returns)<jupyter_output>IDT         KND        OERZ        BWLF         SGN
2008-08-31         nan         nan         nan         nan         nan
2008-09-30  3.13172138  0.72709204  5.76874778  1.77557845  0.04098317
2008-10-31 -3.78816218 -0.67583590 -4.95433863 -1.67093250 -0.24929051
2008-11-30  0.05579709  0.29199789  0.00697116  1.05956179  0.30686995
                   IDT         KND        OERZ        BWLF         SGN
2008-08-31         nan         nan         nan         nan         nan
2008-09-30         nan         nan         nan         nan         nan
2008-10-31  3.13172138  0.72709204  5.76874778  1.77557845  0.04098317
2008-11-30 -3.78816218 -0.67583590 -4.95433863 -1.67093250 -0.24929051
Tests Passed<jupyter_text>View DataLet's get the previous month's and next month's returns.<jupyter_code>prev_returns = shift_returns(monthly_close_returns, 1)
lookahead_returns = shift_returns(monthly_close_returns, -1)

project_helper.plot_shifted_returns(
    prev_returns.loc[:, apple_ticker],
    monthly_close_returns.loc[:, apple_ticker],
    'Previous Returns of {} Stock'.format(apple_ticker))
project_helper.plot_shifted_returns(
    lookahead_returns.loc[:, apple_ticker],
    monthly_close_returns.loc[:, apple_ticker],
    'Lookahead Returns of {} Stock'.format(apple_ticker))<jupyter_output>ticker               A         AAL         AAP        AAPL        ABBV  \
date                                                                     
2013-07-31         nan         nan         nan         nan         nan   
2013-08-31  0.04181412 -0.18015337 -0.02977582  0.08044762 -0.06518370   
2013-09-30  0.09657861  0.15979244  0.03282284 -0.02171531  0.04855545   
2013-10-31 -0.00960698  0.14734639  0.18195865  0.09201927  0.08860637   
2013-11-30  0.05388057  0.06647111  0.01828314  0.06772063  0.00000000   
2013-12-31  0.06769609  0.07267716  0.09197258  0.00886237  0.08616823   
2014-01-31  0.01664682  0.28421071  0.03663543 -0.11394918 -0.06220213   
2014-02-28 -0.02120344  0.09598737  0.10373913  0.05588347  0.03355617   
2014-03-31 -0.01790035 -0.00897599 -0.00629368  0.01975642  0.00957880   
2014-04-30 -0.03182502 -0.04270218 -0.04205794  0.09476126  0.02214224   
2014-05-31  0.05227357  0.13552541  0.02346722  0.07577509  0.04229556   
2014-06-30  0.01103586  0.06739797  0.[...]<jupyter_text>Generate Trading SignalA trading signal is a sequence of trading actions, or results that can be used to take trading actions. A common form is to produce a "long" and "short" portfolio of stocks on each date (e.g. end of each month, or whatever frequency you desire to trade at). This signal can be interpreted as rebalancing your portfolio on each of those dates, entering long ("buy") and short ("sell") positions as indicated.Here's a strategy that we will try:> For each month-end observation period, rank the stocks by _previous_ returns, from the highest to the lowest. Select the top performing stocks for the long portfolio, and the bottom performing stocks for the short portfolio.Implement the `get_top_n` function to get the top performing stock for each month. Get the top performing stocks from `prev_returns` by assigning them a value of 1. For all other stocks, give them a value of 0. For example, using the following `prev_returns`:```                                     Previous Returns               A         B         C         D         E         F         G2013-07-08     0.015     0.082     0.096     0.020     0.075     0.043     0.0742013-07-09     0.037     0.095     0.027     0.063     0.024     0.086     0.025...            ...       ...       ...       ...       ...       ...       ...```The function `get_top_n` with `top_n` set to 3 should return the following:```                                     Previous Returns               A         B         C         D         E         F         G2013-07-08     0         1         1         0         1         0         02013-07-09     0         1         0         1         0         1         0...            ...       ...       ...       ...       ...       ...       ...```*Note: You may have to use Panda's [`DataFrame.iterrows`](https://pandas.pydata.org/pandas-docs/version/0.21/generated/pandas.DataFrame.iterrows.html) with [`Series.nlargest`](https://pandas.pydata.org/pandas-docs/version/0.21/generated/pandas.Series.nlargest.html) in order to implement the function. This is one of those cases where creating a vecorization solution is too difficult.*<jupyter_code>def get_top_n(prev_returns, top_n):
    """
    Select the top performing stocks
    
    Parameters
    ----------
    prev_returns : DataFrame
        Previous shifted returns for each ticker and date
    top_n : int
        The number of top performing stocks to get
    
    Returns
    -------
    top_stocks : DataFrame
        Top stocks for each ticker and date marked with a 1
    """
    # ref: https://stackoverflow.com/questions/38607381/python-pandas-if-the-data-is-nan-then-change-to-be-0-else-change-to-be-1-in-d
    
    # define lambda functions to use
    get_top = lambda x: x.nlargest(top_n)
    do_binary = lambda x: 0 if pd.isnull(x) else 1
    
    # 
    top_stocks = prev_returns.apply(get_top, axis=1).applymap(do_binary)
    print(top_stocks)
    
    return top_stocks

project_tests.test_get_top_n(get_top_n)<jupyter_output>ASOE  DLBE  KPNL  SAVB  ZOT
2008-08-31     0     0     0     0    0
2008-09-30     0     0     0     0    0
2008-10-31     1     1     0     0    1
2008-11-30     0     1     1     1    0
Tests Passed<jupyter_text>View DataWe want to get the best performing and worst performing stocks. To get the best performing stocks, we'll use the `get_top_n` function. To get the worst performing stocks, we'll also use the `get_top_n` function. However, we pass in `-1*prev_returns` instead of just `prev_returns`. Multiplying by negative one will flip all the positive returns to negative and negative returns to positive. Thus, it will return the worst performing stocks.<jupyter_code>top_bottom_n = 50
df_long = get_top_n(prev_returns, top_bottom_n)
df_short = get_top_n(-1*prev_returns, top_bottom_n)
project_helper.print_top(df_long, 'Longed Stocks')
project_helper.print_top(df_short, 'Shorted Stocks')<jupyter_output>A  AAL  AAP  AAPL  ABBV  ABC  ABT  ACN  ADBE  ADI  ...  XEC  XEL  \
date                                                           ...             
2013-07-31  0    0    0     0     0    0    0    0     0    0  ...    0    0   
2013-08-31  0    0    0     0     0    0    0    0     0    0  ...    0    0   
2013-09-30  1    0    0     1     0    0    0    0     0    0  ...    1    0   
2013-10-31  0    1    0     0     0    0    0    0     1    0  ...    1    0   
2013-11-30  0    1    1     0     0    0    0    0     0    0  ...    0    0   
2013-12-31  0    0    0     0     0    0    0    0     0    0  ...    0    0   
2014-01-31  0    0    1     0     1    0    0    0     0    0  ...    1    0   
2014-02-28  0    1    0     0     0    0    0    0     0    0  ...    0    0   
2014-03-31  0    0    0     0     0    0    0    0     1    0  ...    1    0   
2014-04-30  0    0    0     0     0    0    0    0     0    0  ...    0    0   
2014-05-31  0    0    0     1     0    0[...]<jupyter_text>Projected ReturnsIt's now time to check if your trading signal has the potential to become profitable!We'll start by computing the net returns this portfolio would return. For simplicity, we'll assume every stock gets an equal dollar amount of investment. This makes it easier to compute a portfolio's returns as the simple arithmetic average of the individual stock returns.Implement the `portfolio_returns` function to compute the expected portfolio returns. Using `df_long` to indicate which stocks to long and `df_short` to indicate which stocks to short, calculate the returns using `lookahead_returns`. To help with calculation, `n_stocks` is provided as the number of stocks we're investing in a single period.<jupyter_code>def portfolio_returns(df_long, df_short, lookahead_returns, n_stocks):
    """
    Compute expected returns for the portfolio, assuming equal investment in each long/short stock.
    
    Parameters
    ----------
    df_long : DataFrame
        Top stocks for each ticker and date marked with a 1
    df_short : DataFrame
        Bottom stocks for each ticker and date marked with a 1
    lookahead_returns : DataFrame
        Lookahead returns for each ticker and date
    n_stocks: int
        The number number of stocks chosen for each month
    
    Returns
    -------
    portfolio_returns : DataFrame
        Expected portfolio returns for each ticker and date
    """
    returns = lookahead_returns * (df_long - df_short)/ n_stocks
    print(returns)
    
    return returns

project_tests.test_portfolio_returns(portfolio_returns)<jupyter_output>QOPJ         LCV        ZXRG         VHX         KLG
2008-08-31  0.00000000  0.00000000  0.00000000  0.00000000  0.00000000
2008-09-30 -0.00000000 -0.00000000 -0.00000000 -0.00000000 -0.00000000
2008-10-31  0.01859903 -0.09733263  0.00232372  0.00000000 -0.10228998
2008-11-30 -0.41819699  0.00000000 -0.86088613  2.30892279  0.28210892
Tests Passed<jupyter_text>View DataTime to see how the portfolio did.<jupyter_code>expected_portfolio_returns = portfolio_returns(df_long, df_short, lookahead_returns, 2*top_bottom_n)
project_helper.plot_returns(expected_portfolio_returns.T.sum(), 'Portfolio Returns')
# Quick visualization, but keep in mind that returns are discrete.
project_helper.plot_stock(expected_portfolio_returns[apple_ticker], '{} Stock'.format(apple_ticker))<jupyter_output><empty_output><jupyter_text>Statistical Tests Annualized Rate of Return<jupyter_code>expected_portfolio_returns_by_date = expected_portfolio_returns.T.sum().dropna()
portfolio_ret_mean = expected_portfolio_returns_by_date.mean()
portfolio_ret_ste = expected_portfolio_returns_by_date.sem()
portfolio_ret_annual_rate = (np.exp(portfolio_ret_mean * 12) - 1) * 100

print("""
Mean:                       {:.6f}
Standard Error:             {:.6f}
Annualized Rate of Return:  {:.2f}%
""".format(portfolio_ret_mean, portfolio_ret_ste, portfolio_ret_annual_rate))<jupyter_output>Mean:                       0.003012
Standard Error:             0.002135
Annualized Rate of Return:  3.68%<jupyter_text>The annualized rate of return allows you to compare the rate of return from this strategy to other quoted rates of return, which are usually quoted on an annual basis.  T-TestOur null hypothesis ($H_0$) is that the actual mean return from the signal is zero. We'll perform a one-sample, one-sided t-test on the observed mean return, to see if we can reject $H_0$.We'll need to first compute the t-statistic, and then find its corresponding p-value. The p-value will indicate the probability of observing a t-statistic equally or more extreme than the one we observed if the null hypothesis were true. A small p-value means that the chance of observing the t-statistic we observed under the null hypothesis is small, and thus casts doubt on the null hypothesis. It's good practice to set a desired level of significance or alpha ($\alpha$) _before_ computing the p-value, and then reject the null hypothesis if $p < \alpha$.For this project, $\alpha = 0.05$, since it's a common value to use.The `analyze_alpha` function was implemented to perform a t-test on the sample of portfolio returns. The `scipy.stats` module was imported to perform the t-test.Note: [`scipy.stats.ttest_1samp`](https://docs.scipy.org/doc/scipy-1.0.0/reference/generated/scipy.stats.ttest_1samp.html) performs a two-sided test, so divide the p-value by 2 to get 1-sided p-value<jupyter_code>from scipy import stats

def analyze_alpha(expected_portfolio_returns_by_date):
    """
    Perform a t-test with the null hypothesis being that the expected mean return is zero.
    
    Parameters
    ----------
    expected_portfolio_returns_by_date : Pandas Series
        Expected portfolio returns for each date
    
    Returns
    -------
    t_value
        T-statistic from t-test
    p_value
        Corresponding p-value
    """
    t_val, p_val = stats.ttest_1samp(expected_portfolio_returns_by_date, 0)
    #print(t_val, p_val/2)
    
    return t_val, p_val/2

project_tests.test_analyze_alpha(analyze_alpha)<jupyter_output>Tests Passed<jupyter_text>View DataLet's see what values we get with our portfolio.<jupyter_code>t_value, p_value = analyze_alpha(expected_portfolio_returns_by_date)
print("""
Alpha analysis:
 t-value:        {:.3f}
 p-value:        {:.6f}
""".format(t_value, p_value))<jupyter_output>Alpha analysis:
 t-value:        1.410
 p-value:        0.082492
{'fraction_non_alphanumeric': 0.08488768774090648, 'fraction_numerical': 0.148730672101369, 'mean_word_length': 2.3091922005571033, 'pattern_counts': {'":': 0, '<': 57, '<?xml version=': 0, '>': 57, 'https://': 6, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 14, 'word_list_counts': {'cursed_substrings.json': 58, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '457896', 'license': 'MIT', 'n_tokens_mistral': 9191, 'n_tokens_neox': 7232, 'n_words': 2199, 'path': 'trading_with_momentum_project.ipynb', 'repo_name': 'geovani-montoya/Momentum-trading-strategy', 'size': '22571'}