<jupyter_start><jupyter_text>Equal-Weight S&P 500 Index Fund Introduction & Library ImportsThe S&P 500 is the world's most popular stock market index. The largest fund that is benchmarked to this index is the SPDR® S&P 500® ETF Trust. It has more than US$250 billion of assets under management.The goal of this section of the course is to create a Python script that will accept the value of your portfolio and tell you how many shares of each S&P 500 constituent you should purchase to get an equal-weight version of the index fund. Library ImportsThe first thing we need to do is import the open-source software libraries that we'll be using in this tutorial.<jupyter_code># Importing libraries
import numpy as np  # numerical computing library
import pandas as pd # panel data library
import requests     # http requests library
import xlsxwriter   # Excel documents writing library
import math         # mathematical functions library<jupyter_output><empty_output><jupyter_text>Importing Our List of StocksThe next thing we need to do is import the constituents of the S&P 500.These constituents change over time, so in an ideal world you would connect directly to the index provider (Standard & Poor's) and pull their real-time constituents on a regular basis.Paying for access to the index provider's API is outside of the scope of this course. There's a static version of the S&P 500 constituents available here. [Click this link to download them now](https://drive.google.com/file/d/1ZJSpbY69DVckVZlO9cC6KkgfSufybcHN/view?usp=sharing). Move this file into the `starter-files` folder so it can be accessed by other files in that directory.Now it's time to import these stocks to our Jupyter Notebook file.<jupyter_code>stocks = pd.read_csv("input/sp_500_stocks.csv") # Obs: there are 5 stocks that have dual shares, that's why there are 505 stocks
stocks

stocks = pd.read_csv("input/Meat_stocks.csv")

stocks<jupyter_output><empty_output><jupyter_text>To get the `User Agent` just google `my user agent` and copy the string display in the result.To see the `HTML` source code, open the page that you want to scrape and in it use the secondary click andselect `View Page Source`. And to inspect the structure in relation with the page, use also the right click andselect `Inspect`. I also recomend to click on the icon with the square and the small arrow of the `Inspect` API (in thetop left of the `Inspect` interface) to be able to detect where each of the parts of the page is represented in the code.Following the ideas of scraping from [here](https://www.youtube.com/watch?v=PPcgtx0sI2E&list=PLOxtsYaYauiSoHSyjo4uUZhnaWjNYdGfL&index=5&t=169s) Acquiring an API TokenNow it's time to import our IEX Cloud API token. This is the data provider that we will be using throughout this course.API tokens (and other sensitive information) should be stored in a `secrets.py` file that doesn't get pushed to your local Git repository. We'll be using a sandbox API token in this course, which means that the data we'll use is randomly-generated and (more importantly) has no cost associated with it.[Click here](http://nickmccullum.com/algorithmic-trading-python/secrets.py) to download your `secrets.py` file. Move the file into the same directory as this Jupyter Notebook before proceeding.<jupyter_code>from secrets import IEX_CLOUD_API_TOKEN # Sometimes you need to ´restart´ the Jupyter Kernel to run this line<jupyter_output><empty_output><jupyter_text>Making Our First API CallNow it's time to structure our API calls to IEX cloud. We need the following information from the API:* Market capitalization for each stock* Price of each stock<jupyter_code>symbol = "AAPL" # This is the symbol for Apple stocks
symbol = "BRFS"
api_url = f'https://sandbox.iexapis.com/stable/stock/{symbol}/quote?token={IEX_CLOUD_API_TOKEN}' # this is the url for the sandbox api
# The next line could take a few seconds
data = requests.get(api_url).json() # .json() make a type python dictionary object
print(data)
print(type(data))<jupyter_output>{'symbol': 'BRFS', 'companyName': 'BRF S.A. - ADR', 'primaryExchange': 'C OI,SW EGOYNCEA X CKHR.KETNN', 'calculationPrice': 'close', 'open': None, 'openTime': None, 'openSource': 'ioflifca', 'close': None, 'closeTime': None, 'closeSource': 'lafcfoii', 'high': None, 'highTime': 1629724648232, 'highSource': 'naie d y1etcel5rmdi uep', 'low': None, 'lowTime': 1613489242526, 'lowSource': 'i uyme5rn1pedit  caelde', 'latestPrice': 4.3, 'latestSource': 'Close', 'latestTime': 'December 31, 2020', 'latestUpdate': 1650973935414, 'latestVolume': None, 'iexRealtimePrice': 4.257, 'iexRealtimeSize': 103, 'iexLastUpdated': 1660214921519, 'delayedPrice': None, 'delayedPriceTime': None, 'oddLotDelayedPrice': None, 'oddLotDelayedPriceTime': None, 'extendedPrice': None, 'extendedChange': None, 'extendedChangePercent': None, 'extendedPriceTime': None, 'previousClose': 4.32, 'previousVolume': 1633931, 'change': -0.07, 'changePercent': -0.01673, 'volume': None, 'iexMarketPercent': 0.02273742133284901, 'iexVo[...]<jupyter_text>Parsing Our API CallThe API call that we executed in the last code block contains all of the information required to build our equal-weight S&P 500 strategy. With that said, the data isn't in a proper format yet. We need to parse it first.<jupyter_code>price = data['latestPrice']
market_cap = data['marketCap']
print(price)<jupyter_output>4.3<jupyter_text>Adding Our Stocks Data to a Pandas DataFrameThe next thing we need to do is add our stock's price and market capitalization to a pandas DataFrame. Think of a DataFrame like the Python version of a spreadsheet. It stores tabular data.<jupyter_code>my_columns = ['Ticker', 'Price','Market Capitalization', 'Number Of Shares to Buy']
final_dataframe = pd.DataFrame(columns = my_columns)
final_dataframe
final_dataframe.append(
    pd.Series(
        [
            symbol,
            price,
            market_cap,
            'N/A'
        ], index = my_columns
        ), ignore_index = True
)<jupyter_output><empty_output><jupyter_text>Looping Through The Tickers in Our List of StocksUsing the same logic that we outlined above, we can pull data for all S&P 500 stocks and store their data in the DataFrame using a `for` loop.<jupyter_code>final_dataframe = pd.DataFrame(columns=my_columns)
i = 0
for stock in stocks['Ticker'][:5]:
    api_url = f'https://sandbox.iexapis.com/stable/stock/{stock}/quote?token={IEX_CLOUD_API_TOKEN}' # this is the url for the sandbox api
    data = requests.get(api_url).json() # .json() make a type python dictionary object
    final_dataframe = final_dataframe.append(
        pd.Series(
        [
                stock,
                data['latestPrice'],
                data['marketCap'],
                'N/A'
            ], index = my_columns
        ), ignore_index = True
    )
    print(i)
    i=i+1
final_dataframe<jupyter_output><empty_output><jupyter_text>Using Batch API Calls to Improve PerformanceBatch API calls are one of the easiest ways to improve the performance of your code.This is because HTTP requests are typically one of the slowest components of a script.Also, API providers will often give you discounted rates for using batch API calls since they are easier for the API provider to respond to.IEX Cloud limits their batch API calls to 100 tickers per request. Still, this reduces the number of API calls we'll make in this section from 500 to 5 - huge improvement! In this section, we'll split our list of stocks into groups of 100 and then make a batch API call for each group.<jupyter_code># Function sourced from 
# https://stackoverflow.com/questions/312443/how-do-you-split-a-list-into-evenly-sized-chunks
def chunks(lst, n):
    """Yield successive n-sized chunks from lst."""
    for i in range(0, len(lst), n):
        yield lst[i:i + n]
symbol_groups = list(chunks(stocks['Ticker'],100))
symbol_strings =[]
for i in range(0, len(symbol_groups)):
    symbol_strings.append(','.join(symbol_groups[i]))
    #print(symbol_strings[i])
    

final_dataframe = pd.DataFrame(columns = my_columns)

for symbol_string in symbol_strings:
#     print(symbol_strings)
    batch_api_call_url = f'https://sandbox.iexapis.com/stable/stock/market/batch/?types=quote&symbols={symbol_string}&token={IEX_CLOUD_API_TOKEN}'
    data = requests.get(batch_api_call_url).json()
    for symbol in symbol_string.split(','):
          final_dataframe = final_dataframe.append(
              pd.Series(
              [
                  symbol,
                  data[symbol]['quote']['latestPrice'],
                  data[symbol]['quote']['marketCap'],
                  'N/A'
              ], index = my_columns
              ), ignore_index = True
          )

final_dataframe<jupyter_output><empty_output><jupyter_text>Calculating the Number of Shares to BuyAs you can see in the DataFrame above, we stil haven't calculated the number of shares of each stock to buy.We'll do that next.<jupyter_code>portfolio_size = input('Enter the value of your portfolio:')

try:
    val = float(portfolio_size)
    print(val)
except ValueError:
    print('Please enter an integer')
position_size = float(portfolio_size) / len(final_dataframe.index)
for i in range(0, len(final_dataframe['Ticker'])-1):
    final_dataframe.loc[i, 'Number Of Shares to Buy'] = math.floor(position_size / final_dataframe['Price'][i])
final_dataframe<jupyter_output><empty_output><jupyter_text>Formatting Our Excel OutputWe will be using the XlsxWriter library for Python to create nicely-formatted Excel files.XlsxWriter is an excellent package and offers tons of customization. However, the tradeoff for this is that the library can seem very complicated to new users. Accordingly, this section will be fairly long because I want to do a good job of explaining how XlsxWriter works. Initializing our XlsxWriter Object<jupyter_code>writer = pd.ExcelWriter('Recomended_trades_Meat_Stocks_of_NYSE.xlsx',writer='xlsxwriter')
final_dataframe.to_excel(writer, 'Recomended Trades', index=False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08712536094792393, 'fraction_numerical': 0.018122075077168176, 'mean_word_length': 4.573806881243064, 'pattern_counts': {'":': 0, '<': 39, '<?xml version=': 0, '>': 39, 'https://': 6, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '428300', 'license': 'MIT', 'n_tokens_mistral': 3112, 'n_tokens_neox': 2923, 'n_words': 1134, 'path': 'My_equal_weight_S&P_500.ipynb', 'repo_name': 'ArrigoCoen/Algorithmic_Trading', 'size': '10043'}