<jupyter_start><jupyter_text>Historical Shape Indicator (HSI),  Adviser Survey Results These are results from the Adviser Survey Workbook.     These trades were generated by a combination of both  Historical Shape Indicator and Logistic Regression Classifier.  Approximately 81 stocks are studied over a two year span.<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sb
sb.set()
%matplotlib inline
%config InlineBackend.figure_format = 'retina'<jupyter_output><empty_output><jupyter_text>Import Processed Data<jupyter_code>df_srv = pd.read_csv('adviser_survey_102919.csv')
df_srv = df_srv.dropna()
df_srv.head()
df_srv['sharpe'] = df_srv['sharpe'].astype(float)
df_srv.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
Int64Index: 276 entries, 0 to 275
Data columns (total 15 columns):
symbol        276 non-null object
equity        276 non-null object
test_year     276 non-null int64
price         276 non-null float64
lowT          276 non-null float64
highT         276 non-null float64
trades        276 non-null int64
in_days       276 non-null int64
t-val         276 non-null float64
p-val         276 non-null float64
exp_ret       276 non-null float64
volatility    276 non-null float64
years         276 non-null float64
sharpe        276 non-null float64
beat          276 non-null float64
dtypes: float64(10), int64(3), object(2)
memory usage: 34.5+ KB<jupyter_text>Survey Analysis<jupyter_code># stock tickers
tickers = df_srv['symbol'].unique().tolist()
tickers.sort()
print(tickers)
# company names 
equities = df_srv['equity'].unique().tolist()
equities.sort()
print(equities)
len(equities)<jupyter_output><empty_output><jupyter_text>Test Years<jupyter_code># test year histogram
years = df_srv['test_year']
plt.figure(figsize=(8,6))
plt.xlabel("Test Year")
plt.ylabel("Frequency")
plt.title("Test Years of Case Studies")
plt.xticks(np.arange(2002, 2020, step=2))
plt.hist(years,18,(2002,2019), color = (.1,.1,.95,.3));<jupyter_output><empty_output><jupyter_text>Stock Nominal Year Price Histogram<jupyter_code># stock price histogram
prices = df_srv['price']
plt.figure(figsize=(8,6))
plt.xlabel("Stock Price USD$")
plt.ylabel("Frequency")
plt.title("Stock Prices for Test Cases")
plt.hist(prices.dropna(),30,(0,250), color = (.1,.1,.95,.3));<jupyter_output><empty_output><jupyter_text>Small portfolios necessarily need small stock prices.  This is to manage the risk of ruin.<jupyter_code># price statistics
prices.describe()<jupyter_output><empty_output><jupyter_text>Indicator Threshold Values<jupyter_code># low threshold histogram
lowTs = df_srv['lowT']
plt.figure(figsize=(8,6))
plt.xlabel("Threshold Value")
plt.ylabel("Frequency")
plt.title("HSI Low Trade Thresholds")
plt.hist(lowTs.dropna(),50,(0,1), color = (.1,.1,.95,.3));<jupyter_output><empty_output><jupyter_text>This graphis shows that one size does not fit all.The optimized tuning varies with the Test case.<jupyter_code>lowTs.describe()
# High threshold histogram
highTs = df_srv['highT']
plt.figure(figsize=(8,6))
plt.xlabel("Threshold Value")
plt.ylabel("Frequency")
plt.title("HSI High Trade Thresholds")
plt.hist(highTs.dropna(),50,(0,1), color = (.1,.1,.95,.3));<jupyter_output><empty_output><jupyter_text>The high threshold seems to mirror image of the low threshold.<jupyter_code>highTs.describe()<jupyter_output><empty_output><jupyter_text>Number of Trades in Test Year<jupyter_code># High threshold histogram
num_trades = df_srv['trades']
plt.figure(figsize=(8,6))
plt.xlabel("Quantity")
plt.ylabel("Frequency")
plt.title("Number of Trades in Test Year")
plt.hist(num_trades.dropna(),50,(0,50), color = (.1,.1,.95,.3));<jupyter_output><empty_output><jupyter_text>This project is not intended for day trading.  The number of trades are typically one every two months in any one market.<jupyter_code># trade count statistics
num_trades.describe()<jupyter_output><empty_output><jupyter_text>In The Market Trade Days<jupyter_code># Number of days in the market histogram
in_days = df_srv['in_days']
plt.figure(figsize=(8,6))
plt.xlabel("Quantity")
plt.ylabel("Frequency")
plt.title("Number of Days In Market During Test Year")
plt.hist(in_days.dropna(),20,(0,252), color = (.1,.1,.95,.3));<jupyter_output><empty_output><jupyter_text>The long trading style is a "buy and hold" strategy.  A day trader might push up against the high quantity  side.<jupyter_code>in_days.describe()
# Days per Trade histogram
days_per_trade = df_srv['in_days']/df_srv['trades']
plt.figure(figsize=(8,6))
plt.xlabel("Days")
plt.ylabel("Frequency")
plt.title("Days per Trade for Test Cases")
plt.hist(days_per_trade.dropna(),20,(0,40), color = (.1,.1,.95,.3));<jupyter_output><empty_output><jupyter_text>The trade duration supports the idea of a couple trades per month.<jupyter_code>days_per_trade.describe()<jupyter_output><empty_output><jupyter_text>T-Test ResultsThese are the results of the Equal Sample Unequal Variance Two-Tail Test.  In this case the null hypothessi relates to no difference between returns,  in the market using the trade rules, and out of the market using trade rules.  The test samples for minimum counts between the two, then samples the larger  case for equal samples.In the case where no significance is found, the indicator plays less of a factor.  In the case where the p-value is small, the indicator plays a greater role.<jupyter_code># T Values histogram
t_vals = df_srv['t-val']
plt.figure(figsize=(8,6))
plt.xlabel("statistic")
plt.ylabel("Frequency")
plt.title("T-Test T-Values")
plt.hist(t_vals.dropna(),20,(-5,5), color = (.1,.1,.95,.3));
t_vals.describe()<jupyter_output><empty_output><jupyter_text>The t-values measure size of the difference relative to variation in the data.  As t increases, the null hypothesis becomes less likely.  That these are peaking off zero, supports some value added by the indicator.<jupyter_code># P Values histogram
p_vals = df_srv['p-val']
plt.figure(figsize=(8,6))
plt.xlabel("statistic")
plt.ylabel("Frequency")
plt.title("T-Test P-Values")
plt.hist(p_vals.dropna(),20,(0,1), color = (.1,.1,.95,.3));
p_vals.describe()<jupyter_output><empty_output><jupyter_text>The statistics indicate that the lower quartile is coming in at 0.065;  this suggests something significant going on with ~1/4 of the group.  Unfortunately, with an average days in the market of 135,   that's also the number of samples per t-test here.  Such a low number of samples may not be sufficient for detecting  return differences of interest.In any case, as the p-value drops might suggest episodes where  in the market trades gave greater return than not; it's not entirely futile.  Expected Returns  The log returns are converted to simple percentages.  In most cases, the test case spans one year.  Since 2019 is partial, the returns for those are lower.<jupyter_code># P Values histogram
exp_rets = df_srv[df_srv['exp_ret']<125]['exp_ret']
plt.figure(figsize=(8,6))
plt.axvline(x=0., color = 'r')
plt.xlabel("Percent")
plt.ylabel("Frequency")
plt.title("Expected Returns in % over Test Case Span")
plt.hist(exp_rets.dropna(),25,(-25,100), color = (.1,.1,.95,.3));
exp_rets.describe()<jupyter_output><empty_output><jupyter_text>In the big picture, a practical approach to stock trading seems this:  win more times than you lose, "cut your losses and let your winners run."The expected returns histogram here supports this idea.  Trade Volatility<jupyter_code># P Values histogram
exp_rets = df_srv[df_srv['volatility']<1]['volatility']
plt.figure(figsize=(8,6))
plt.xlabel("Percent")
plt.ylabel("Frequency")
plt.title("Volatility")
plt.hist(exp_rets.dropna(),20,(0,.35), color = (.1,.1,.95,.3));
df_srv[df_srv['volatility']<1]['volatility'].describe()<jupyter_output><empty_output><jupyter_text>Sharpe RatioThis reveals degrees of health in the trades,   higher as better and too high as suspicious.<jupyter_code># Sharpe ratio histogram
sharpes = df_srv[df_srv['sharpe']<10]['sharpe']
plt.figure(figsize=(8,6))
plt.axvline(x=1.,color='r')
plt.xlabel("Ratio")
plt.ylabel("Frequency")
plt.title("Sharpe Ratios for Test Cases")
plt.hist(sharpes.dropna(),20,(-2,8), color = (.1,.1,.95,.3));
sharpes.describe()<jupyter_output><empty_output><jupyter_text>A sharpe ratio >1 is considered good.  That the mean value exceeds 1 supports the idea many trades are healthy.   Market BenchmarkThe difference to the S&P500 indicate the percentage the trades beat the market.  This comparison is stock to benchmark on similar days "in the market."  As most test cases are one year terms, the percentage indicated is roughly per year.<jupyter_code># Beat Percentage histogram
beats = df_srv[df_srv['beat']<125]['beat']
plt.figure(figsize=(8,6))
plt.axvline(x=0,color='r')
plt.xlabel("Percent")
plt.ylabel("Frequency")
plt.title("Beat The Market Percentages for Test Cases")
plt.hist(beats.dropna(),50,(-50,100), color = (.1,.1,.95,.3));
beats.describe()<jupyter_output><empty_output><jupyter_text>The lower quartile is within 2% of even.  This indicates that ~3/4 of the cases beat the market.  On average, beating the market by 13% is expected.  In the worst case, it was 26% draw down.    Ticker Rank Sorting  The idea here is that some stocks likely complement this trade approach better than others.  We aim to do "Cherry Picking" on the "Cherry Picked", for improving out chances.  We want "the best bang for the buck" so to speak.    Sharpe ratios are used for this pursuit because they factor volatility risks.<jupyter_code>df_srv = df_srv.sort_values(by=['sharpe'], ascending=False)
df_rnk = df_srv[df_srv['sharpe']<10].copy()
df_recent = df_rnk[df_rnk['test_year']>2017]
df_recent.head(30)<jupyter_output><empty_output><jupyter_text>Many of the top 30 entries from 2018 and 2019 exhibit (relatively) low p-values.  Also it seems like 2019 is a good year for stock trading.    Efficiency Frontier<jupyter_code># Trade Efficiency Frontier
data_ef = df_rnk[((df_rnk['volatility']<1)&(df_rnk['exp_ret']<100))&(df_rnk['price']<150)]
plt.figure(figsize=(12, 8))
g = sb.relplot(x='volatility', y='exp_ret', 
               hue = 'price', size = 'beat', 
               sizes=(5, 50), alpha=.5, 
               palette="plasma", data = data_ef)
g.set(title = 'Expected Returns vs. Volatility')
g.set(yticks=np.arange(-50, 100, step=25))
g.set(xticks=np.arange(0, .4, step=0.1));<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1270712909441233, 'fraction_numerical': 0.034393063583815026, 'mean_word_length': 5.820630749014454, 'pattern_counts': {'":': 0, '<': 98, '<?xml version=': 0, '>': 91, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '22', 'dir': 'jupyter-structured-clean-dedup', 'id': '595460', 'license': 'MIT', 'n_tokens_mistral': 3832, 'n_tokens_neox': 3453, 'n_words': 1001, 'path': 'Class Stage/adviser_analysis_102919.ipynb', 'repo_name': 'patronical/Historical_Shape_Indicator', 'size': '10380'}