<jupyter_start><jupyter_text>@author: <NAME>  All needed impots<jupyter_code>import os
import numpy as np
from datetime import datetime, date, timedelta

import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline

from sklearn.preprocessing import MinMaxScaler

from tensorflow import keras
from tensorflow.keras.layers import Bidirectional, Dropout, Activation, Dense, LSTM
from tensorflow.python.keras.layers import CuDNNLSTM
from tensorflow.keras.models import Sequential

!pip install yfinance
import yfinance as yf<jupyter_output>Collecting yfinance
  Downloading https://files.pythonhosted.org/packages/7a/e8/b9d7104d3a4bf39924799067592d9e59119fcfc900a425a12e80a3123ec8/yfinance-0.1.55.tar.gz
Requirement already satisfied: pandas>=0.24 in /usr/local/lib/python3.6/dist-packages (from yfinance) (1.1.5)
Requirement already satisfied: numpy>=1.15 in /usr/local/lib/python3.6/dist-packages (from yfinance) (1.19.5)
Requirement already satisfied: requests>=2.20 in /usr/local/lib/python3.6/dist-packages (from yfinance) (2.23.0)
Requirement already satisfied: multitasking>=0.0.7 in /usr/local/lib/python3.6/dist-packages (from yfinance) (0.0.9)
Collecting lxml>=4.5.1
[?25l  Downloading https://files.pythonhosted.org/packages/bd/78/56a7c88a57d0d14945472535d0df9fb4bbad7d34ede658ec7961635c790e/lxml-4.6.2-cp36-cp36m-manylinux1_x86_64.whl (5.5MB)
[K     |████████████████████████████████| 5.5MB 18.4MB/s 
[?25hRequirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas>=0.24->yfinance) (20[...]<jupyter_text>Downloading Bitcoin dataframe with yahoo finance API and then plotting this DF**reset_index** method is used only to make normal numerical indexes, instead of *Date* beeing an index<jupyter_code>df = yf.download('BTC-USD', start='2015-08-07')
df = df.reset_index()
df
ax = df.plot(x='Date', y='Close');
ax.set_xlabel("Date")
ax.set_ylabel("Close Price (USD)")<jupyter_output><empty_output><jupyter_text>NormalizationSince the scaler expects the data to be shaped as (x, y), I'm adding a dummy dimension using reshape before applying it. **reshape** method transforms array of *Closed* values from shape (2001, 1) into (2001, 1)<jupyter_code>scaler = MinMaxScaler()

close_price = df.Close.values.reshape(-1, 1)

scaled_close = scaler.fit_transform(close_price)

scaled_close.shape, np.isnan(scaled_close).any()<jupyter_output><empty_output><jupyter_text>Preprocessing LSTMs expect the data to be in 3 dimensions. We need to split the data into sequences of some preset length. The shape we want to obtain is: **[batch_size, sequence_length, n_features]** The process of building sequences works by creating a sequence of a specified length at position 0. Then it's shifted one position to the right (e.g. 1) and creates another sequence. The process is repeated until all possible positions are used. Here, *SEQ_LEN* parameter is set to 20, which means, that 20 days are predicting the next one<jupyter_code>SEQ_LEN = 20
whole_data = np.empty([1, 20, 1])
whole_targets = np.empty([1, 1])

def to_sequences(data, seq_len):
    d = []

    for index in range(len(data) - seq_len + 1):
        d.append(data[index: index + seq_len])

    return np.array(d)

def preprocess(data_raw, seq_len, train_split):

    global whole_data, whole_targets
    whole_data = to_sequences(data_raw, seq_len)
    whole_targets = whole_data[:, -1, :]
    print('whole_data: ', whole_data.shape)
    print('whole_targets: ', whole_targets.shape)

    num_train = int(train_split * whole_data.shape[0])

    X_train = whole_data[:num_train, :, :]
    y_train = whole_data[:num_train, -1, :]

    X_test = whole_data[num_train:, :, :]
    y_test = whole_data[num_train:, -1, :]

    return X_train, y_train, X_test, y_test

X_train, y_train, X_test, y_test = preprocess(scaled_close, SEQ_LEN, train_split = 0.95)

X_train.shape, X_test.shape<jupyter_output>whole_data:  (1987, 20, 1)
whole_targets:  (1987, 1)<jupyter_text>Simply checking arrays if everything's alright<jupyter_code># Last 20 scaled values of the DataFrame
scaled_close[-20:], scaled_close.shape
print('The last 2 sequences from whole sequenced data and it\'s shape: \n {} | {} \n Last 5 values to predict (simple values from DF) : \n \
    {}'.format(whole_data[-2:], whole_data.shape, whole_targets[-5:]))
X_test[-1], X_test.shape
y_test[-5:], y_test.shape<jupyter_output><empty_output><jupyter_text>ModelBidirectional RNNs allows us to train on the sequence data in forward and backward (reversed) direction. In practice, this approach works well with LSTMs.The output layer has a single neuron (predicted Bitcoin price).<jupyter_code>DROPOUT = 0.1
WINDOW_SIZE = SEQ_LEN 

model = keras.Sequential()

model.add(Bidirectional(LSTM(WINDOW_SIZE, return_sequences=True),
                        input_shape=(WINDOW_SIZE, X_train.shape[-1])))
model.add(Dropout(rate=DROPOUT))

model.add(Bidirectional(LSTM((WINDOW_SIZE * 2), return_sequences=True)))
model.add(Dropout(rate=DROPOUT))

model.add(Bidirectional(LSTM(WINDOW_SIZE, return_sequences=False)))

model.add(Dense(units=1))

model.add(Activation('linear'))

model.compile(
    loss='mean_squared_error', 
    optimizer='adam'
)

model.summary()<jupyter_output>Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
bidirectional (Bidirectional (None, 20, 40)            3520      
_________________________________________________________________
dropout (Dropout)            (None, 20, 40)            0         
_________________________________________________________________
bidirectional_1 (Bidirection (None, 20, 80)            25920     
_________________________________________________________________
dropout_1 (Dropout)          (None, 20, 80)            0         
_________________________________________________________________
bidirectional_2 (Bidirection (None, 40)                16160     
_________________________________________________________________
dense (Dense)                (None, 1)                 41        
________________________________________________________[...]<jupyter_text>Training<jupyter_code>BATCH_SIZE = 64
EPOCHS = 120

history = model.fit(
    X_train, 
    y_train, 
    epochs=EPOCHS, 
    batch_size=BATCH_SIZE, 
    shuffle=False,
    validation_split=0.1
)
model.evaluate(X_test, y_test)
plt.figure(figsize = (18,12))
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()<jupyter_output><empty_output><jupyter_text>Prediction on Test Data<jupyter_code>y_hat = model.predict(X_test)

y_test_inverse = scaler.inverse_transform(y_test)
y_hat_inverse = scaler.inverse_transform(y_hat)

plt.figure(figsize = (18,12))
plt.plot(y_test_inverse, label="Actual Price", color='green')
plt.plot(y_hat_inverse, label="Predicted Price", color='red')
 
plt.title('Ethereum price prediction')
plt.xlabel('Time [days]')
plt.ylabel('Price')
plt.legend(loc='best')

plt.show();<jupyter_output><empty_output><jupyter_text>Checking some real (*y_test_inverse*) and predicted *y_hat_inverse* values<jupyter_code>y_test_inverse[85:]
y_hat_inverse[85:]
df.tail(15)<jupyter_output><empty_output><jupyter_text>Now, predicting into the future.Since 20 values are predicting the next one and there is no value for tomorrow to make prediction upon, I'm gonna make an artificial dummy values for that. I'm making a new sequence, where the last value will be repeated twice (inserting a dummy data) : if the last value for today is *[0.94239681]*, then this value will be set for tomorrowPrediction array will be : pred_arrayTarget arrays will be:  targ_array<jupyter_code>def get_prediction_array(X):
  """
  Inserts an other array to the end of the whole_data sequence in order to make
  array for predicting tomorrow's value

  returns: array of shape (None, 20, 1)
  """
  next = len(X)
  a = np.insert(X,[next],X[next-1], axis=0)
  for i_ in range(len(a[100]) - 1):
    if(i_ == 20):
      a[next][i_][0] == a[next][i_ - 1][0]
    else:
      a[next][i_][0] = a[next][i_ + 1][0]
  
  return a
pred_array = get_prediction_array(whole_data)
pred_array.shape
pred_array[-2:], pred_array.shape
targ_array = get_prediction_array(whole_targets)
targ_array.shape 
y_hat = model.predict(pred_array)

y_true_inverse = scaler.inverse_transform(targ_array)
y_hat_inverse = scaler.inverse_transform(y_hat)

plt.figure(figsize = (18,12))
plt.plot(y_true_inverse, label="Actual Price", color='green')
plt.plot(y_hat_inverse, label="Predicted Price", color='red')
 
plt.title('Ethereum price prediction')
plt.xlabel('Time')
plt.ylabel('Price')
plt.legend(loc='best')
 
plt.show();
print('Predicted Ethereum price for the next day: {} and predicted Ethereum price for the next day: {}'.format(y_hat_inverse[-1], y_true_inverse[-1]))<jupyter_output>Predicted Ethereum price for the next day: [38445.3] and predicted Ethereum price for the next day: [37508.75]<jupyter_text>Making a dataset of data-predicted-true values**NOTE** - prediction starts from 20th **(SEQ_LEN)** day from *2015-08-07* (the first day data was received inf DF from).This means, that the first *SEQ_LEN = 20* values do not have any prediction at all<jupyter_code># number of rows in DF
num_data_points = df.index.stop
# where the sequence prediction started from
started_ = num_data_points - len(y_true_inverse) + 1
tomorrow = date.today() + timedelta(days=1)
tomorrow = tomorrow.strftime('%Y-%m-%d')

ds = []

for i_ in df.Date[started_:]:
  ds.append(i_.strftime('%Y-%m-%d'))

ds.append(tomorrow)

print(len(ds), ds[-2:])<jupyter_output>1988 ['2021-02-05', '2021-02-06']<jupyter_text>Last *true* value in **forecast** DataFrame is set to zero<jupyter_code>y_true = y_true_inverse.reshape(len(y_true_inverse),)
y_true[-1] = 0
y_pred = y_hat_inverse.reshape(len(y_hat_inverse),)

d = {"ds": ds, "true": y_true, 'pred' : y_pred}
forecast = pd.DataFrame(d)
forecast
df<jupyter_output><empty_output><jupyter_text>Saving the model<jupyter_code>file_name = 'BidirectLSTM_BTC-1f.h5'
model.save(file_name)
print("Saved model `{}` to disk".format(file_name))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12700871248789933, 'fraction_numerical': 0.03659244917715392, 'mean_word_length': 5.400867410161091, 'pattern_counts': {'":': 2, '<': 49, '<?xml version=': 0, '>': 57, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 2}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '433644', 'license': 'MIT', 'n_tokens_mistral': 3849, 'n_tokens_neox': 3463, 'n_words': 846, 'path': 'models/lstm/bidirect_lstm/BidirectLSTM_BTC_1f.ipynb', 'repo_name': 'kvlsky/interactive-bitcoin-price-prediction', 'size': '10336'}