<jupyter_start><jupyter_text>Credit: https://github.com/luke4u/Time_Series_Forecasting/blob/main/Stock_price_prediction/Stock_price_prediction_LSTM.ipynb?fbclid=IwAR0tT9LIaM3BGtmu3kJkVIdjxNiBjLOwXl-b8V7m-xgjgK2B4hkCZjbOX-Q<jupyter_code>!nvidia-smi
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
from sklearn.preprocessing import MinMaxScaler<jupyter_output><empty_output><jupyter_text>The data is saved in .csv files. 2004-2019  Download dataset<jupyter_code>!wget https://github.com/kaopanboonyuen/Kao/raw/master/dataset/GOOG.csv
data = pd.read_csv('GOOG.csv', date_parser = True)
data.sample(5)
data.info()
plt.figure(figsize=(18, 8))
plt.plot(data['Open'])
plt.title("Google Stock Prices")
plt.xlabel("Time (oldest -> latest)")
plt.ylabel("Stock Opening Price")
plt.show()<jupyter_output><empty_output><jupyter_text>Data preparation  Train/test split<jupyter_code>data_training = data[data['Date']<'2019-01-01'].copy()
data_test = data[data['Date']>='2019-01-01'].copy()
data_training = data_training.drop(['Date', 'Adj Close'], axis = 1)
data_training.head()<jupyter_output><empty_output><jupyter_text>The next step is to scale the stock prices between (0, 1) to avoid intensive computation. Common methods include Standardization and Normalization  Normalize data<jupyter_code>scaler = MinMaxScaler()
data_training = scaler.fit_transform(data_training)
scaler.inverse_transform(data_training)
scale = 1/scaler.scale_
temp = (data_training * scale) + scaler.min_
temp<jupyter_output><empty_output><jupyter_text>create RNN with 60 timesteps, i.e. look 60 previous time steps<jupyter_code>data_training[0:10]<jupyter_output><empty_output><jupyter_text>A special data structure is needed to cover 60-time stamps, based on which RNN will predict the 61st price.<jupyter_code>X_train = []
y_train = []

for i in range(60, data_training.shape[0]):
    X_train.append(data_training[i-60:i])
    y_train.append(data_training[i, 0])

X_train, y_train = np.array(X_train), np.array(y_train)
X_train.shape<jupyter_output><empty_output><jupyter_text>Building Model1: LSTM<jupyter_code>import tensorflow as tf
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense, LSTM, Dropout
tf.__version__<jupyter_output><empty_output><jupyter_text>Fundamentally, we are building a NN regressor for continuous value prediction using LSTM. First, initialize the model.<jupyter_code>modelLSTM = Sequential()

modelLSTM.add(LSTM(units = 60, activation = 'relu', return_sequences = True, input_shape = (X_train.shape[1], 5)))
modelLSTM.add(Dropout(0.2))

modelLSTM.add(LSTM(units = 60, activation = 'relu', return_sequences = True))
modelLSTM.add(Dropout(0.2))

modelLSTM.add(LSTM(units = 80, activation = 'relu', return_sequences = True))
modelLSTM.add(Dropout(0.2))

modelLSTM.add(LSTM(units = 120, activation = 'relu'))
modelLSTM.add(Dropout(0.2))

modelLSTM.add(Dense(units = 1))
modelLSTM.summary()<jupyter_output>Model: "sequential_2"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 lstm_4 (LSTM)               (None, 60, 60)            15840     
                                                                 
 dropout_8 (Dropout)         (None, 60, 60)            0         
                                                                 
 lstm_5 (LSTM)               (None, 60, 60)            29040     
                                                                 
 dropout_9 (Dropout)         (None, 60, 60)            0         
                                                                 
 lstm_6 (LSTM)               (None, 60, 80)            45120     
                                                                 
 dropout_10 (Dropout)        (None, 60, 80)            0         
                                                      [...]<jupyter_text>Callbacks*   EarlyStoping: It will stop the traning if score of model didn't increase. This prevent model from overfitting. We are to set max in 10 epoches if it didn't increase then we will stop the training*   ReduceLROnPlateau: Use for reduce the learning rate. In 3 steps the score didn't increase we will reduce the learning rate to improve the training*   ModelCheckpoint: Use for save model only when the score increased<jupyter_code>from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint, ReduceLROnPlateau
#create callback
filepath = 'my_best_modelLSTM_yourself.hdf5'
checkpoint = ModelCheckpoint(filepath=filepath, 
                             monitor='loss',
                             verbose=0, 
                             save_best_only=True,
                             mode='min')

earlystopping = EarlyStopping(
    monitor='loss', 
    patience=3, 
    min_delta=0, 
    mode='auto'
)

reduce_lr = ReduceLROnPlateau(
    monitor='loss', 
    factor=0.2,   
    patience=2, 
    min_lr=0.001,
    verbose=0
)
callbacks = [checkpoint, earlystopping, reduce_lr]<jupyter_output><empty_output><jupyter_text>For optimizer, we use Adam, a safe choice to start with. The loss function is the mean of squared errors between actual values and predictions.<jupyter_code>modelLSTM.compile(optimizer='adam', loss = 'mean_squared_error')<jupyter_output><empty_output><jupyter_text>RNN weights are updated every 32 stock prices with a batch size of 32.<jupyter_code># history = modelLSTM.fit(X_train, y_train, epochs=10, batch_size=32, callbacks=callbacks)
# download pre-trained weight
!wget https://storage.googleapis.com/kao-weights/my_best_modelLSTM.hdf5
history= modelLSTM.load_weights('my_best_modelLSTM.hdf5')<jupyter_output><empty_output><jupyter_text>Testing<jupyter_code>data = pd.read_csv('GOOG.csv', date_parser = True)
data_training = data[data['Date']<'2019-01-01'].copy()
data_test = data[data['Date']>='2019-01-01'].copy()
data_test.head()
data_training.tail(60)<jupyter_output><empty_output><jupyter_text>Prepare testing data<jupyter_code>past_60_days = data_training.tail(60)
df = past_60_days.append(data_test, ignore_index = True)
df = df.drop(['Date', 'Adj Close'], axis = 1)
df.head()
inputs = scaler.transform(df)
inputs
X_test = []
y_test = []

for i in range(60, inputs.shape[0]):
    X_test.append(inputs[i-60:i])
    y_test.append(inputs[i, 0])
X_test, y_test = np.array(X_test), np.array(y_test)
X_test.shape, y_test.shape<jupyter_output><empty_output><jupyter_text>Model prediction<jupyter_code>y_pred = modelLSTM.predict(X_test)<jupyter_output><empty_output><jupyter_text>Don’t forget, what we have predicted is the scaled values, so we need to reverse the prediction.<jupyter_code>scaler.scale_
scale = 1/scaler.scale_[0]
scale
y_pred = y_pred*scale
y_test = y_test*scale
import math
from sklearn.metrics import mean_squared_error

def MAPE(Y_actual,Y_Predicted):
    mape = np.mean(np.abs((Y_actual - Y_Predicted)/Y_actual))*100
    return mape

print( 'MPAE =', MAPE(y_test, y_pred) )
print( 'RMSE =', math.sqrt(mean_squared_error(y_test, y_pred)) )
# Visualising the results
plt.figure(figsize=(14,5))
plt.plot(y_test, color = 'red', label = 'Real Google Stock Price')
plt.plot(y_pred, color = 'blue', label = 'Predicted Google Stock Price')
plt.title('Google Stock Price Prediction Using LSTM')
plt.xlabel('Time')
plt.ylabel('Google Stock Price')
plt.legend()
plt.show()<jupyter_output><empty_output><jupyter_text>Building Model2: GRU<jupyter_code>from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense, GRU, Dropout
# The GRU architecture
modelGRU = Sequential()

modelGRU.add(GRU(units = 50, activation = 'relu', return_sequences = True, input_shape=(X_train.shape[1],5)))
modelGRU.add(Dropout(0.2))

modelGRU.add(GRU(units=50, return_sequences=True))
modelGRU.add(Dropout(0.2))

modelGRU.add(GRU(units=50, return_sequences=True))
modelGRU.add(Dropout(0.2))

modelGRU.add(GRU(units=50))
modelGRU.add(Dropout(0.2))

modelGRU.add(Dense(units=1))
modelGRU.summary()
#create callback
filepath = 'my_best_modelGRU.hdf5'
checkpoint = ModelCheckpoint(filepath=filepath, 
                             monitor='loss',
                             verbose=0, 
                             save_best_only=True,
                             mode='min')

earlystopping = EarlyStopping(
    monitor='loss', 
    patience=3, 
    min_delta=0, 
    mode='auto'
)

reduce_lr = ReduceLROnPlateau(
    monitor='loss', 
    factor=0.2,   
    patience=2, 
    min_lr=0.001,
    verbose=0
)
callbacks = [checkpoint, earlystopping, reduce_lr]
modelGRU.compile(optimizer='sgd', loss='mean_squared_error')
# modelGRU.fit(X_train, y_train, epochs=10, batch_size=16, callbacks=callbacks)
# download pre-trained weight
!wget https://storage.googleapis.com/kao-weights/my_best_modelGRU.hdf5
history= modelGRU.load_weights('my_best_modelGRU.hdf5')
y_pred = modelGRU.predict(X_test)
y_pred = y_pred*scale
import math
from sklearn.metrics import mean_squared_error
print( 'MPAE =', MAPE(y_test, y_pred) )
print( 'RMSE =', math.sqrt(mean_squared_error(y_test, y_pred)) )
# Visualising the results
plt.figure(figsize=(14,5))
plt.plot(y_test, color = 'red', label = 'Real Google Stock Price')
plt.plot(y_pred, color = 'blue', label = 'Predicted Google Stock Price')
plt.title('Google Stock Price Prediction Using GRU')
plt.xlabel('Time')
plt.ylabel('Google Stock Price')
plt.legend()
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12104204172402838, 'fraction_numerical': 0.02478026051043101, 'mean_word_length': 3.9341692789968654, 'pattern_counts': {'":': 0, '<': 66, '<?xml version=': 0, '>': 67, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 15, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '454519', 'license': 'MIT', 'n_tokens_mistral': 3379, 'n_tokens_neox': 3146, 'n_words': 642, 'path': 'code/week6_dl_2/StockPricePredication_LSTM_v4.ipynb', 'repo_name': 'stangthanin/2110446_DataScience_2021s2', 'size': '9443'}