<jupyter_start><jupyter_text>Analyzing stock for investing in a long term.

Project where I acquired data from [FinViz](https://finviz.com/), [Polygon.io](https://polygon.io/) and [TradingView](https://www.tradingview.com/) in order to check stocks with a growth perspective using simple FinViz' filters. Then I chose the best ones out of scanned stocks, compared companies with other companies from the same industry and committed a sentiment analysis and completed a portfolio allocation.<jupyter_code>author = '<NAME>'
author_url = 'https://www.linkedin.com/in/nikolaimelnikov/'<jupyter_output><empty_output><jupyter_text>Installing necessary libraries in Colab<jupyter_code>!pip install -U git+https://github.com/mariostoev/finviz
!pip install yfinance
!pip install alpaca_trade_api
!pip install newsapi-python
!pip install flair<jupyter_output><empty_output><jupyter_text>Importing necessary libraries<jupyter_code>import pandas as pd
import numpy as np

import finviz
from finviz.screener import Screener
import yfinance as yf
from pandas_datareader import data as pdr
yf.pdr_override() 

from datetime import timedelta
from datetime import date

import nest_asyncio
nest_asyncio.apply()

import matplotlib.pyplot as plt
import seaborn as sns
sns.set_theme(style="whitegrid")<jupyter_output><empty_output><jupyter_text>Choosing stocks with the growth potential. 


Most important financial indicators for long term: P/E under 35, P/B under 2 and P/Cashflow under 25<jupyter_code>filters_long = ['cap_smallover','fa_curratio_o0.5','fa_debteq_u1','fa_eps5years_pos','fa_epsyoy_o10','fa_estltgrowth_pos','fa_pe_u35','fa_ps_u5','fa_roe_o10','fa_pb_u2']  
stock_list_long = Screener(filters=filters_long, table='Overview', order='price') 
stock_list_long.to_csv("stocks_long.csv")
stocks_long = pd.read_csv('stocks_long.csv', index_col=['Ticker'])<jupyter_output><empty_output><jupyter_text>Most important for growth are a positive ROE and stock not being oversold.<jupyter_code>filters_growth = ['fa_debteq_u1','fa_eps5years_pos','fa_estltgrowth_pos','fa_pe_profitable','fa_ps_u5','fa_roe_o10','sh_avgvol_o300','sh_price_o10','ta_rsi_nos40','fa_pfcf_u20','fa_div_o1','fa_pb_u3']  
stock_list_growth = Screener(filters=filters_growth, table='Overview', order='price') 
stock_list_growth.to_csv("stocks_growth.csv")
stocks_growth = pd.read_csv('stocks_growth.csv', index_col=['Ticker'])
stocks = pd.concat([stocks_long, stocks_growth])
stocks.drop_duplicates(inplace=True)<jupyter_output><empty_output><jupyter_text>Most occurring Sectors between the stocks that have a perspective to growth.<jupyter_code>sns.barplot(y=stocks.Sector.value_counts().index, x=stocks.Sector.value_counts())
stocks.drop(['No.','Company','Country','Change', 'Sector', 'Volume'], inplace=True, axis=1)
stocks.head(10)
#Dictionary for finviz industries
industry_dict = {
    'Oil & Gas Midstream': 'ind_oilgasmidstream',
    'Gold':'ind_gold',
    'Scientific & Technical Instruments':'ind_scientifictechnicalinstruments',
    'Apparel Manufacturing':'ind_apparelmanufacturing',
    'Drug Manufacturers - Specialty & Generic':'ind_drugmanufacturersspecialtygeneric',
    'Household & Personal Products':'ind_householdpersonalproducts',
    'Software - Application':'ind_softwareapplication',
    'Education & Training Services':'ind_educationtrainingservices',
    'Chemicals':'ind_chemicals',
    'Staffing & Employment Services':'ind_staffingemploymentservices',
    'Packaging & Containers':'ind_packagingcontainers',
    'Asset Management':'ind_assetmanagement',
    'Diagnostics & Research':'ind_diagnosticsresearch',
    'Communication Equipment':'ind_communicationequipment',
    'Pollution & Treatment Controls':'ind_pollutiontreatmentcontrols',
    'Electronic Components':'ind_electroniccomponents',
    'Apparel Retail':'ind_apparelretail',
    'Lumber & Wood Production':'ind_lumberwoodproduction',
    'Medical Devices':'ind_medicaldevices',
    'Drug Manufacturers - General':'ind_drugmanufacturersgeneral',
    'Biotechnology':'ind_biotechnology',
    'Consulting Services':'ind_consultingservices',
    'Beverages - Non-Alcoholic':'ind_beveragesnonalcoholic',
    'Farm & Heavy Construction Machinery':'ind_farmheavyconstructionmachinery',
    'Computer Hardware':'ind_computerhardware',
    'Internet Content & Information':'ind_internetcontentinformation',
    'Specialty Business Services':'ind_specialtybusinessservices',
    'Software - Infrastructure':'ind_softwareinfrastructure',
    'Semiconductor Equipment & Materials':'ind_semiconductorequipmentmaterials',
    'Electronic Gaming & Multimedia':'ind_electronicgamingmultimedia',
    'Security & Protection Services':'ind_securityprotectionservices',
    'Personal Services':'ind_personalservices',
    'Internet Retail':'ind_internetretail',
    'Financial Data & Stock Exchanges':'ind_financialdatastockexchanges',
    'Medical Instruments & Supplies':'ind_medicalinstrumentssupplies',
    'Health Information Services':'ind_healthinformationservices',
    'Credit Services':'ind_creditservices',
    'Information Technology Services':'ind_informationtechnologyservices',
    'Beverages - Brewers':'ind_beveragesbrewers',
    'Engineering & Construction':'ind_engineeringconstruction',
    'Capital Markets':'ind_capitalmarkets',
    'Insurance - Diversified':'ind_insurancediversified',
    'Specialty Industrial Machinery':'ind_specialtyindustrialmachinery',
    'Medical Distribution':'ind_medicaldistribution',
    'Building Products & Equipment':'ind_buildingproductsequipment',
    'Marine Shipping': 'ind_marineshipping',
    'Utilities - Diversified': 'ind_utilitiesdiversified',
    'Grocery Stores': 'ind_grocerystores',
    'Trucking':'ind_trucking',
    'Medical Care Facilities':'ind_medicalcarefacilities',
    'Auto Parts':'ind_autoparts',
    'Furnishings, Fixtures & Appliances':'ind_furnishingsfixturesappliances',
    'Residential Construction':'ind_residentialconstruction',
    'Industrial Distribution':'ind_industrialdistribution',
    'Insurance Brokers':'ind_insurancebrokers',
    'Packaged Foods':'ind_packagedfoods',
    'Airports & Air Services':'ind_airportsairservices',
    'Semiconductors':'ind_semiconductors',
    'Publishing':'ind_publishing', 
    'Electronics & Computer Distribution':'ind_electronicscomputerdistribution',
    'Real Estate Services':'ind_realestateservices',
    'Auto & Truck Dealerships':'ind_autotruckdealerships',
    'Recreational Vehicles':'ind_recreationalvehicles',
    'Building Materials':'ind_buildingmaterials',
    'Electrical Equipment & Parts':'ind_electricalequipmentparts',
    'Specialty Retail':'ind_specialtyretail',
    'Insurance - Life':'ind_insurancelife',
    'Food Distribution':'ind_fooddistribution',
    'Metal Fabrication':'ind_metalfabrication',
    'Business Equipment & Supplies':'ind_businessequipmentsupplies',
    'Integrated Freight & Logistics':'ind_integratedfreightlogistics',
    'Tools & Accessories':'ind_toolsaccessories',
    'Discount Stores':'ind_discountstores', 
    'Leisure':'ind_leisure',
    'Aerospace & Defense':'ind_aerospacedefense',
    'REIT - Office':'ind_reitoffice',
    'Footwear & Accessories':'ind_footwearaccessories',
    'Home Improvement Retail':'ind_homeimprovementretail',
    'Railroads':'ind_railroads',
    'Telecom Services':'ind_telecomservices',
    'Entertainment':'ind_entertainment',
    'Broadcasting' : 'ind_broadcasting',
    'Steel' : 'ind_steel',
    'Other Industrial Metals & Mining': 'ind_otherindustrialmetalsmining',
    'Banks - Regional': 'ind_banksregional',
    'Insurance - Specialty': 'ind_insurancespecialty',
    'Mortgage Finance': 'ind_mortgagefinance',
    'REIT - Specialty': 'ind_reitspecialty',
    'Rental & Leasing Services': 'ind_rentalleasingservices',
    'Healthcare Plans': 'ind_healthcareplans',
    'Farm Products': 'ind_farmproducts',
    'Banks - Diversified': 'ind_banksdiversified',
    'Insurance - Property & Casualty': 'ind_insurancepropertycasualty'
}<jupyter_output><empty_output><jupyter_text>Finding good performing competitors for each company from the stocks dataframe.<jupyter_code>competitors = []
i = 0
for row in stocks.index:
  ind = industry_dict[stocks['Industry'].iloc[i]]
  i += 1
  filters_competitor = [ind, 'cap_smallover','fa_pe_o5','sh_price_o80','ta_rsi_nos40']  
  competitors_list = []
  try:
    stock_list_competitor = Screener(filters=filters_competitor, table='Overview', order='price') 
    for stock in stock_list_competitor: 
      competitors_list.append(stock['Ticker'])
  except Exception:
    pass
  if not competitors_list:
    competitors.append('-')
  else:
    competitors.append(competitors_list)
  
stocks['Competitors'] = competitors<jupyter_output><empty_output><jupyter_text>Selecting companies with with a low price of a stock and that have good-performing competitors<jupyter_code>stocks_to_analyze = stocks[(stocks['Competitors'] != '-')&(stocks['Price'] <= 100)]
stocks_to_analyze.head(10)<jupyter_output><empty_output><jupyter_text>Replace Competotors' tickers with price change in years and see their performance for the past 3 years.<jupyter_code>results = pd.DataFrame(columns=['Start','End','Difference'])
results_row = pd.DataFrame(columns=['Avg'])
for row in stocks_to_analyze.index:
  for competitor in stocks_to_analyze.Competitors[row]:
    data = pdr.get_data_yahoo(competitor, start=date.today()-timedelta(days=365*3), end=date.today())
    temp = pd.DataFrame(
        {
            'Start': data.Open.iloc[0],
            'End': data.Open.iloc[-1],
            'Difference': data.Open.iloc[-1]/data.Open.iloc[0]
        }, index=[competitor]
    )
    results = pd.concat([results, temp])
  temp_row = pd.DataFrame(
      {
          'Avg':results.Difference.mean()
      }, index = [row]
  )
  results_row = pd.concat([results_row, temp_row])
stocks_to_analyze.drop('Competitors', axis=1, inplace=True)
stocks_to_analyze['competitors_performance'] = results_row
stocks_to_analyze.head(10)<jupyter_output><empty_output><jupyter_text>Analyzing company's performance and comparing it to other companies. Here we analyze how the price of company's competitors has changed during the 3-year-long period. Positive difference means that the company underperformed competitors and vice versa.<jupyter_code>results_ticker = pd.DataFrame(columns=['Difference'])
for ticker in stocks_to_analyze.index:
  data = pdr.get_data_yahoo(ticker, start=date.today()-timedelta(days=365*3), end=date.today())
  temp = pd.DataFrame(
       {
           'Difference': data.Open.iloc[-1]/data.Open.iloc[0]
       }, index=[ticker]
   )
  results_ticker = pd.concat([results_ticker, temp])
stocks_to_analyze['companys_increase'] = results_ticker
stocks_to_analyze['difference_with_industry'] = stocks_to_analyze['competitors_performance']-stocks_to_analyze['companys_increase']
stocks_to_analyze.head(10)<jupyter_output><empty_output><jupyter_text>Sentiment Analysis

Getting news abut the companies and conducting a sentiment analysis to see if the news are positive or negative. Positivity of the news is import in the long term.<jupyter_code>import alpaca_trade_api as tradeapi
from newsapi import NewsApiClient
from flair.data import Sentence
import torch
import flair
api = tradeapi.REST(f'{polygon}',
                    'https://api.polygon.io')


def sentiment(stock, api):
    url = 'https://www.tradingview.com/screener/'
    flair_sentiment = flair.models.TextClassifier.load('en-sentiment')
    newsapi = NewsApiClient(api_key=f'{newsapikey}')
    response = newsapi.get_everything(qintitle=stock)
    news = api.polygon.news(stock)
    file = open('news.txt', 'w')

    sentiment = 0
    for line in response['articles']:
        words = str(line['title'])
        file.write(words)
        sentence = Sentence(str(words))
        flair_sentiment.predict(sentence)
        total_sentiment = sentence.labels

        if total_sentiment[0].value == 'NEGATIVE':
            sentiment -= total_sentiment[0].to_dict()['confidence'] 
        elif total_sentiment[0].value == 'POSITIVE':
            sentiment += total_sentiment[0].to_dict()['confidence']

    for source in news:
        words = source.summary
        try:
            file.write(words)
        except:
            print('FAILSAFE ACTIVATED')
        file.write('\n')
        sentence = Sentence(str(words))
        try:
            flair_sentiment.predict(sentence)
        except:
            print("\n")
        total_sentiment = sentence.labels
        if total_sentiment[0].value == 'NEGATIVE':
          total_sentiment[0].to_dict()['confidence']
        if total_sentiment[0].value == 'POSITIVE':
            total_sentiment[0].to_dict()['confidence']
            sentiment += total_sentiment[0].to_dict()['confidence']

    file.close()
    return sentiment

sentiments = pd.DataFrame(columns=['Sentiment'])
import time
for ticker in stocks_to_analyze.index:
  x = sentiment(ticker, api)
  temp = pd.DataFrame(
       {
           'Sentiment': x
       }, index=[ticker]
   )
  sentiments = pd.concat([sentiments, temp])
  time.sleep(15)
stocks_to_analyze['Sentiment'] = sentiments<jupyter_output><empty_output><jupyter_text>Setting total multiplier that relys on competitors' performance and company's sentiment<jupyter_code>stocks_to_analyze['difference_multiplier'] = ''
stocks_to_analyze['sentiment_multiplier'] = ''
difference_sent = stocks_to_analyze['Sentiment'].mean()
difference = stocks_to_analyze['difference_with_industry'].mean()
for ticker in stocks_to_analyze.index:
  if (stocks_to_analyze['difference_with_industry'].loc[ticker] > difference).all():
    stocks_to_analyze['difference_multiplier'][ticker] =  2
  elif ((stocks_to_analyze['difference_with_industry'].loc[ticker] < difference) & (stocks_to_analyze['difference_with_industry'].loc[ticker] > 0)).all():
    stocks_to_analyze['difference_multiplier'][ticker] =  1
  else:
    stocks_to_analyze['difference_multiplier'][ticker] = 0.5
  
  if (stocks_to_analyze['Sentiment'].loc[ticker] > difference_sent).all():
    stocks_to_analyze['sentiment_multiplier'][ticker] =  2
  elif ((stocks_to_analyze['Sentiment'].loc[ticker] < difference_sent) & (stocks_to_analyze['Sentiment'].loc[ticker] > 0)).all():
    stocks_to_analyze['sentiment_multiplier'][ticker] =  1
  else:
    stocks_to_analyze['sentiment_multiplier'][ticker] = 0.5

stocks_to_analyze['total_multiplier'] = stocks_to_analyze['sentiment_multiplier'] * stocks_to_analyze['difference_multiplier']
stocks_to_analyze.drop(['difference_multiplier','sentiment_multiplier'], axis=1, inplace=True)
stocks_to_analyze
plt.figure(figsize=(10,5))
ax = stocks_to_analyze.Sentiment.sort_values(ascending=False).plot(kind='bar')
ax.axhline(stocks_to_analyze.Sentiment.mean(), color='red', linewidth=2, label='Mean')
plt.text(30,22,'Mean')
ax.set(ylabel='Sentiment points', title='Ticker\'s Sentiment')
plt.show()<jupyter_output><empty_output><jupyter_text>Portfolio allocation<jupyter_code>total_money = 100000  #initial funds are equal $100.000
total_mult = stocks_to_analyze.total_multiplier.sum()
stocks_to_analyze['money_to_place'] = ''
stocks_to_analyze['stocks_to_buy'] = ''
for row in stocks_to_analyze.index:
  stocks_to_analyze['money_to_place'][row] = (total_money * (stocks_to_analyze['total_multiplier'][row]/total_mult) )  
  stocks_to_analyze['stocks_to_buy'][row] = (stocks_to_analyze['money_to_place'][row] / stocks_to_analyze['Price'][row])
stocks_to_analyze[['money_to_place', 'stocks_to_buy']]
fig, ax1 = plt.subplots(figsize=(12, 12))
sns.barplot(x=stocks_to_analyze.index, hue='money_to_place', y='stocks_to_buy', data=stocks_to_analyze, ax=ax1)
ax1.legend(title='Money to place in a stock')
ax1.set(xlabel='Ticker')
plt.xticks(rotation=90)
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12147843430565944, 'fraction_numerical': 0.007604088755921216, 'mean_word_length': 5.433440256615878, 'pattern_counts': {'":': 0, '<': 57, '<?xml version=': 0, '>': 58, 'https://': 7, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '375953', 'license': 'MIT', 'n_tokens_mistral': 5451, 'n_tokens_neox': 5117, 'n_words': 866, 'path': 'Long_Term_Investing.ipynb', 'repo_name': 'MelnNik/Long-Term-Portfolio', 'size': '16054'}