<jupyter_start><jupyter_text>Crypto ArbitrageIn this Challenge, you'll take on the role of an analyst at a high-tech investment firm. The vice president (VP) of your department is considering arbitrage opportunities in Bitcoin and other cryptocurrencies. As Bitcoin trades on markets across the globe, can you capitalize on simultaneous price dislocations in those markets by using the powers of Pandas?For this assignment, you’ll sort through historical trade data for Bitcoin on two exchanges: Bitstamp and Coinbase. Your task is to apply the three phases of financial analysis to determine if any arbitrage opportunities exist for Bitcoin.This aspect of the Challenge will consist of 3 phases.1. Collect the data.2. Prepare the data.3. Analyze the data.    Import the required libraries and dependencies.<jupyter_code>import pandas as pd
from pathlib import Path
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>Collect the DataTo collect the data that you’ll need, complete the following steps:Instructions. 1. Using the Pandas `read_csv` function and the `Path` module, import the data from `bitstamp.csv` file, and create a DataFrame called `bitstamp`. Set the DatetimeIndex as the Timestamp column, and be sure to parse and format the dates.2. Use the `head` (and/or the `tail`) function to confirm that Pandas properly imported the data.3. Repeat Steps 1 and 2 for `coinbase.csv` file.  Step 1: Using the Pandas `read_csv` function and the `Path` module, import the data from `bitstamp.csv` file, and create a DataFrame called `bitstamp`. Set the DatetimeIndex as the Timestamp column, and be sure to parse and format the dates.<jupyter_code>bitstamp_df = pd.read_csv(Path('Resources/bitstamp.csv'), # Imported bitstamp.csv
    index_col = "Timestamp",
    parse_dates = True,
    infer_datetime_format = True
)<jupyter_output><empty_output><jupyter_text>Step 2: Use the `head` (and/or the `tail`) function to confirm that Pandas properly imported the data.<jupyter_code>bitstamp_df.head() # Testing DataFrame by showing first 5 values.<jupyter_output><empty_output><jupyter_text>Step 3: Repeat Steps 1 and 2 for `coinbase.csv` file.<jupyter_code>coinbase_df = pd.read_csv(Path('Resources/coinbase.csv'),  # Imported coinbase.csv
    index_col = "Timestamp",
    parse_dates = True,
    infer_datetime_format = True
)
coinbase_df.head() # Testing DataFrame by showing first 5 values.<jupyter_output><empty_output><jupyter_text>Prepare the DataTo prepare and clean your data for analysis, complete the following steps:1. For the bitstamp DataFrame, replace or drop all `NaN`, or missing, values in the DataFrame.2. Use the `str.replace` function to remove the dollar signs ($) from the values in the Close column.3. Convert the data type of the Close column to a `float`.4. Review the data for duplicated values, and drop them if necessary.5. Repeat Steps 1–4 for the coinbase DataFrame.  Step 1: For the bitstamp DataFrame, replace or drop all `NaN`, or missing, values in the DataFrame.<jupyter_code>bitstamp_df = bitstamp_df.dropna().copy() # Dropping null data cells.<jupyter_output><empty_output><jupyter_text>Step 2: Use the `str.replace` function to remove the dollar signs ($) from the values in the Close column.<jupyter_code>bitstamp_df["Close"] = bitstamp_df["Close"].str.replace('$', '', regex=False).copy() # Removing $ symbols from the DataFrame.<jupyter_output><empty_output><jupyter_text>Step 3: Convert the data type of the Close column to a `float`.<jupyter_code>bitstamp_df["Close"] = bitstamp_df["Close"].astype("float") # Transformed the close column to a float variable.<jupyter_output><empty_output><jupyter_text>Step 4: Review the data for duplicated values, and drop them if necessary.<jupyter_code>bitstamp_df.duplicated() # Reviewing duplicates.
bitstamp_df = bitstamp_df.drop_duplicates().copy() # Dropping duplicates.<jupyter_output><empty_output><jupyter_text>Step 5: Repeat Steps 1–4 for the coinbase DataFrame.<jupyter_code>coinbase_df = coinbase_df.dropna().copy() # Dropping null data cells.
coinbase_df["Close"] = coinbase_df["Close"].str.replace('$', '', regex=False).copy() # Removing $ symbols from the DataFrame.
coinbase_df["Close"] = coinbase_df["Close"].astype("float") # Transformed the close column to a float variable.
coinbase_df.duplicated() # Reviewing duplicates.
coinbase_df = coinbase_df.drop_duplicates().copy() # Dropping duplicates.<jupyter_output><empty_output><jupyter_text>Analyze the DataYour analysis consists of the following tasks: 1. Choose the columns of data on which to focus your analysis.2. Get the summary statistics and plot the data.3. Focus your analysis on specific dates.4. Calculate the arbitrage profits.  Step 1: Choose columns of data on which to focus your analysis.Select the data you want to analyze. Use `loc` or `iloc` to select the following columns of data for both the bitstamp and coinbase DataFrames:* Timestamp (index)* Close<jupyter_code>bitstamp_sliced = bitstamp_df.iloc[:,[3]] # Eliminating unnecessary columns
bitstamp_sliced.head()
coinbase_sliced = coinbase_df.iloc[:,[3]] # Eliminating unnecessary columns
coinbase_sliced.head()<jupyter_output><empty_output><jupyter_text>Step 2: Get summary statistics and plot the data.Sort through the time series data associated with the bitstamp and coinbase DataFrames to identify potential arbitrage opportunities. To do so, complete the following steps:1. Generate the summary statistics for each DataFrame by using the `describe` function.2. For each DataFrame, create a line plot for the full period of time in the dataset. Be sure to tailor the figure size, title, and color to each visualization.3. In one plot, overlay the visualizations that you created in Step 2 for bitstamp and coinbase. Be sure to adjust the legend and title for this new visualization.4. Using the `loc` and `plot` functions, plot the price action of the assets on each exchange for different dates and times. Your goal is to evaluate how the spread between the two exchanges changed across the time period that the datasets define. Did the degree of spread change as time progressed?<jupyter_code>bitstamp_sliced.describe(include='all') # Generating summary stats
coinbase_sliced.describe(include='all') # Generating summary stats
bitstamp_sliced.plot(figsize=(15,7), rot=90, color="red") # Plotting Bitstamp BTC prices.
plt.title('Bitstamp BTC Closing Prices')
plt.xlabel('Date')
plt.ylabel('Price')
plt.legend(['Bitstamp Price'])
plt.show()
coinbase_sliced.plot(figsize=(15,7), rot=90, color="blue") # Plotting Coinbase BTC prices.
plt.title('Coinbase BTC Closing Prices')
plt.xlabel('Date')
plt.ylabel('Price')
plt.legend(['Coinbase Price'])
plt.show()
ax = bitstamp_sliced.plot(figsize=(15,7), rot=90, color="red") # Plotting Bitstamp vs Coinbase BTC prices.
coinbase_sliced.plot(ax=ax, rot=90, color="blue")
plt.title('Bitstamp vs Coinbase BTC Closing Prices')
plt.xlabel('Date')
plt.ylabel('Price')
plt.legend(['Bitstamp BTC Price','Coinbase BTC Price'])
plt.ylim(5000,18000)
plt.show()
first_month_bitstamp = bitstamp_sliced.loc['2018-01-01':'2018-02-01'] # Plotting the first month of Bitstamp vs Coinbase BTC prices.
first_month_coinbase = coinbase_sliced.loc['2018-01-01':'2018-02-01']
ax = first_month_bitstamp.plot(figsize=(15,8), rot=90, color="red")
first_month_coinbase.plot(ax=ax, rot=90, color="blue")
plt.title('Bitstamp vs Coinbase BTC Closing Prices (01/01/21 - 02/01/21)')
plt.xlabel('Date')
plt.ylabel('Price')
plt.legend(['Bitstamp BTC Price','Coinbase BTC Price'])
plt.ylim(8000,18000)
plt.show()
last_month_bitstamp = bitstamp_sliced.loc['2018-03-01':'2018-04-01'] # Plotting the last month of Bitstamp vs Coinbase BTC prices.
last_month_coinbase = coinbase_sliced.loc['2018-03-01':'2018-04-01']
ax = last_month_bitstamp.plot(figsize=(15,8), rot=90, color="red")
last_month_coinbase.plot(ax=ax, rot=90, color="blue")
plt.title('Bitstamp vs Coinbase BTC Closing Prices (03/01/21 - 04/01/21)')
plt.xlabel('Date')
plt.ylabel('Price')
plt.legend(['Bitstamp BTC Price','Coinbase BTC Price'])
plt.ylim(6000,16000)
plt.show()<jupyter_output><empty_output><jupyter_text>**Question** Based on the visualizations of the different time periods, has the degree of spread change as time progressed?**Answer** The data suggests that the degree of spread between BTC price listings on Bitstamp and Coinbase has decreased over the 4-month timeframe. As seen in the two figures above (each representing one month of BTC price differential between Bitstamp and Coinbase), price spreads are much larger during the first month of data when compared to the final month of data. If we analyze the month of January (top plot), we can see many small price variations during dips and peaks between the dates 01/01/21 and 01/16/21. Once again, between the dates 01-26-21 and 01-20-21, we see a multiday period where the Bitstamp BTC price listing was significantly higher than the price listed on Coinbase.Examining the second plot containing the final month of price data, we can see that the data displays a negligible amount of price spread between the dates 03-02-21 and 03-06-21, but this variation is far lower than the first month's data. The rest of the final month data exhibits near-parity between the prices found on either exchange.These two plots both contain a price range of 10,000 dollars. The minimum and maximum prices are different, but the equal range allows us to compare the plots without any incongruencies or exhaggerations in price differences.Short Answer: Yes, there is a noticeable decrease in price spread as time moves forward.  Step 3: Focus Your Analysis on Specific DatesFocus your analysis on specific dates by completing the following steps:1. Select three dates to evaluate for arbitrage profitability. Choose one date that’s early in the dataset, one from the middle of the dataset, and one from the later part of the time period.2. For each of the three dates, generate the summary statistics and then create a box plot. This big-picture view is meant to help you gain a better understanding of the data before you perform your arbitrage calculations. As you compare the data, what conclusions can you draw?<jupyter_code>first_day_bitstamp = bitstamp_sliced.loc['2018-01-02 00:00':'2018-01-02 23:59'] # Plotting the first selected day of Bitstamp vs Coinbase BTC prices.
first_day_coinbase = coinbase_sliced.loc['2018-01-02 00:00':'2018-01-02 23:59']
ax = first_day_bitstamp.plot(figsize=(15,8), rot=90, color="red")
first_day_coinbase.plot(ax=ax, rot=90, color="blue")
plt.title('Bitstamp vs Coinbase BTC Closing Prices (01/02/18)')
plt.xlabel('Time')
plt.ylabel('Price')
plt.legend(['Bitstamp BTC Price','Coinbase BTC Price'])
plt.ylim(13000,16000)
plt.show()
arbitrage_spread_early = first_day_coinbase - first_day_bitstamp # Generating summary stats for the first day arbitrage spread.
arbitrage_spread_early.describe(include="all")
arbitrage_spread_early.plot(kind="box") # Generating box plot for the first day arbitrage spread.
plt.title('BTC Arbitrage Spread for 01/02/18')
plt.ylabel('Price Difference')
plt.show()
middle_day_bitstamp = bitstamp_sliced.loc['2018-01-28 00:00':'2018-01-28 23:59'] # Plotting the second selected day of Bitstamp vs Coinbase BTC prices.
middle_day_coinbase = coinbase_sliced.loc['2018-01-28 00:00':'2018-01-28 23:59']
ax = middle_day_bitstamp.plot(figsize=(15,8), rot=90, color="red")
middle_day_coinbase.plot(ax=ax, rot=90, color="blue")
plt.title('Bitstamp vs Coinbase BTC Closing Prices (01/28/18)')
plt.xlabel('Time')
plt.ylabel('Price')
plt.legend(['Bitstamp BTC Price','Coinbase BTC Price'])
plt.ylim(10000,13000)
plt.show()
arbitrage_spread_middle = middle_day_bitstamp - middle_day_coinbase # Generating summary stats for the second day arbitrage spread.
arbitrage_spread_middle.describe(include="all")
arbitrage_spread_middle.plot(kind="box") # Generating box plot for the second day arbitrage spread.
plt.title('BTC Arbitrage Spread for 01/28/18')
plt.ylabel('Price Difference')
plt.show()
late_day_bitstamp = bitstamp_sliced.loc['2018-03-28 00:00':'2018-03-28 23:59'] # Plotting the third selected day of Bitstamp vs Coinbase BTC prices.
late_day_coinbase = coinbase_sliced.loc['2018-03-28 00:00':'2018-03-28 23:59']
ax = late_day_bitstamp.plot(figsize=(15,8), rot=90, color="red")
late_day_coinbase.plot(ax=ax, rot=90, color="blue")
plt.title('Bitstamp vs Coinbase BTC Closing Prices (03/28/18)')
plt.xlabel('Time')
plt.ylabel('Price')
plt.legend(['Bitstamp BTC Price','Coinbase BTC Price'])
plt.ylim(7500,8500)
plt.show()
arbitrage_spread_late = late_day_bitstamp - late_day_coinbase # Generating summary stats for the third day arbitrage spread.
arbitrage_spread_late.describe(include="all")
arbitrage_spread_late.plot(kind="box") # Generating box plot for the third day arbitrage spread.
plt.title('BTC Arbitrage Spread for 03/28/18')
plt.ylabel('Price Difference')
plt.show()<jupyter_output><empty_output><jupyter_text>Step 4: Calculate the Arbitrage ProfitsCalculate the potential profits for each date that you selected in the previous section. Your goal is to determine whether arbitrage opportunities still exist in the Bitcoin market. Complete the following steps:1. For each of the three dates, measure the arbitrage spread between the two exchanges by subtracting the lower-priced exchange from the higher-priced one. Then use a conditional statement to generate the summary statistics for each arbitrage_spread DataFrame, where the spread is greater than zero.2. For each of the three dates, calculate the spread returns. To do so, divide the instances that have a positive arbitrage spread (that is, a spread greater than zero) by the price of Bitcoin from the exchange you’re buying on (that is, the lower-priced exchange). Review the resulting DataFrame.3. For each of the three dates, narrow down your trading opportunities even further. To do so, determine the number of times your trades with positive returns exceed the 1% minimum threshold that you need to cover your costs.4. Generate the summary statistics of your spread returns that are greater than 1%. How do the average returns compare among the three dates?5. For each of the three dates, calculate the potential profit, in dollars, per trade. To do so, multiply the spread returns that were greater than 1% by the cost of what was purchased. Make sure to drop any missing values from the resulting DataFrame.6. Generate the summary statistics, and plot the results for each of the three DataFrames.7. Calculate the potential arbitrage profits that you can make on each day. To do so, sum the elements in the profit_per_trade DataFrame.8. Using the `cumsum` function, plot the cumulative sum of each of the three DataFrames. Can you identify any patterns or trends in the profits across the three time periods?(NOTE: The starter code displays only one date. You'll want to do this analysis for two additional dates).  1. For each of the three dates, measure the arbitrage spread between the two exchanges by subtracting the lower-priced exchange from the higher-priced one. Then use a conditional statement to generate the summary statistics for each arbitrage_spread DataFrame, where the spread is greater than zero.*NOTE*: For illustration, only one of the three dates is shown in the starter code below.<jupyter_code>arbitrage_spread_early = first_day_coinbase - first_day_bitstamp # Generating summary stats for the first day arbitrage spread.
arbitrage_spread_early.describe(include="all")
arbitrage_spread_middle = middle_day_bitstamp - middle_day_coinbase # Generating summary stats for the second day arbitrage spread.
arbitrage_spread_middle.describe(include="all")
arbitrage_spread_late = late_day_bitstamp - late_day_coinbase # Generating summary stats for the third day arbitrage spread.
arbitrage_spread_late.describe(include="all")<jupyter_output><empty_output><jupyter_text>2. For each of the three dates, calculate the spread returns. To do so, divide the instances that have a positive arbitrage spread (that is, a spread greater than zero) by the price of Bitcoin from the exchange you’re buying on (that is, the lower-priced exchange). Review the resulting DataFrame.<jupyter_code>spread_return_early = arbitrage_spread_early[arbitrage_spread_early > 0] / first_day_bitstamp # Generated spread return for the first day arbitrage spread.
spread_return_early.describe(include="all").round(5)
spread_return_middle = arbitrage_spread_middle[arbitrage_spread_middle > 0] / middle_day_coinbase # Generated spread return for the second day arbitrage spread.
spread_return_middle.describe(include="all").round(5)
spread_return_late = arbitrage_spread_late[arbitrage_spread_late > 0] / late_day_coinbase # Generated spread return for the third day arbitrage spread.
spread_return_late.describe(include="all").round(5)<jupyter_output><empty_output><jupyter_text>3. For each of the three dates, narrow down your trading opportunities even further. To do so, determine the number of times your trades with positive returns exceed the 1% minimum threshold that you need to cover your costs.<jupyter_code>profitable_trades_early = spread_return_early[spread_return_early > 0.01] # Generated the amount of profitable trades over 1% on first day.
profitable_trades_early.dropna().head(10)
profitable_trades_middle = spread_return_middle[spread_return_middle > 0.01] # Generated the amount of profitable trades over 1% on second day.
profitable_trades_middle.dropna().head(10)
profitable_trades_late = spread_return_late[spread_return_late > 0.01] # Generated the amount of profitable trades over 1% on third day.
profitable_trades_late.dropna().head(10)<jupyter_output><empty_output><jupyter_text>4. Generate the summary statistics of your spread returns that are greater than 1%. How do the average returns compare among the three dates?<jupyter_code>profitable_trades_early[profitable_trades_early > 0.01].describe(include="all") # Generated the summary stats of profitable trades over 1% on first day.
profitable_trades_middle[profitable_trades_middle > 0.01].describe(include="all") # Generated the summary stats of profitable trades over 1% on second day.
profitable_trades_late[profitable_trades_late > 0.01].describe(include="all") # Generated the summary stats of profitable trades over 1% on third day.<jupyter_output><empty_output><jupyter_text>5. For each of the three dates, calculate the potential profit, in dollars, per trade. To do so, multiply the spread returns that were greater than 1% by the cost of what was purchased. Make sure to drop any missing values from the resulting DataFrame.<jupyter_code>profit_early = profitable_trades_early * first_day_bitstamp # Generated profit amounts for profitable trades over 1% on first day.
profit_per_trade_early = profit_early.dropna()
profit_per_trade_early.head()<jupyter_output><empty_output><jupyter_text>6. Generate the summary statistics, and plot the results for each of the three DataFrames.<jupyter_code>profit_per_trade_early.describe(include="all") # Generated summary stats for profit amounts for profitable trades over 1% on first day.
profit_per_trade_early.plot(kind="bar", rot=0) # Generated box plot for profit amounts for profitable trades over 1% on first day.
plt.title('Profit Opportunities Over 1% (01/02/18)')
plt.xlabel('Date + Time')
plt.ylabel('Profit')
plt.legend([])
plt.show() # I chose a bar figure for this day since there were only 2 values above 1%.<jupyter_output><empty_output><jupyter_text>7. Calculate the potential arbitrage profits that you can make on each day. To do so, sum the elements in the profit_per_trade DataFrame.<jupyter_code>total_profits_early = profit_per_trade_early.sum() # Generated the sum of profit amounts for profitable trades over 1% on first day.
total_profits_early
profit_middle = profitable_trades_middle * middle_day_coinbase  # Generated profit amounts for profitable trades over 1% on second day.
profit_per_trade_middle = profit_middle.dropna()
profit_per_trade_middle
profit_per_trade_middle.describe(include="all") # Generated summary stats for profit amounts for profitable trades over 1% on second day.
profit_per_trade_middle.plot() # Generated line plot for profit amounts for profitable trades over 1% on second day.
plt.title('Profit Opportunities Over 1% (01/28/18)')
plt.xlabel('Date + Time')
plt.ylabel('Profit')
plt.legend([])
plt.show()
total_profits_middle = profit_per_trade_middle.sum() # Generated the sum of profit amounts for profitable trades over 1% on second day.
total_profits_middle # Total profits on 1/28/18
profit_late = profitable_trades_late * late_day_coinbase # Generated profit amounts for profitable trades over 1% on third day. Found none.
profit_per_trade_late = profit_late.dropna()
profit_per_trade_late
# No need to generate summary data since there are no profit chances over 1% on day 3.
# The late day (2018-03-28) that was selected had no opportunities for arbitrage above 1%, therefore a plot of this day is unnecessary.<jupyter_output><empty_output><jupyter_text>8. Using the `cumsum` function, plot the cumulative sum of each of the three DataFrames. Can you identify any patterns or trends in the profits across the three time periods?<jupyter_code>cumulative_profit_early = profit_per_trade_early.cumsum() # Generated the cumulative profit amounts for profitable trades over 1% on first day.
cumulative_profit_early.plot(rot=45) # Plotted the cumulative profit amounts for profitable trades over 1% on first day
plt.title('Cumulative Profit (01/02/18)')
plt.xlabel('Date + Time')
plt.ylabel('Profit')
plt.legend([])
plt.show()
cumulative_profit_middle = profit_per_trade_middle.cumsum() # Generated the cumulative profit amounts for profitable trades over 1% on second day.
cumulative_profit_middle.plot(rot=45) # Plotted the cumulative profit amounts for profitable trades over 1% on second day.
plt.title('Cumulative Profit (01/28/18)')
plt.xlabel('Date + Time')
plt.ylabel('Profit')
plt.legend([])
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08366985797903716, 'fraction_numerical': 0.02649847996733064, 'mean_word_length': 6.007949125596184, 'pattern_counts': {'":': 0, '<': 81, '<?xml version=': 0, '>': 90, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '20', 'dir': 'jupyter-structured-clean-dedup', 'id': '577520', 'license': 'Unlicense', 'n_tokens_mistral': 7027, 'n_tokens_neox': 6269, 'n_words': 2487, 'path': 'Starter_Code/crypto_arbitrage.ipynb', 'repo_name': 'MaxAcheson/Module_3_Challenge', 'size': '22039'}