<jupyter_start><jupyter_text>IRBT Stock Prices Prediction Make sure to import all the libraries needed through out the application<jupyter_code>import os
import pandas as pd
import urllib.request, json
import numpy as np
import tensorflow as tf

from datetime import datetime<jupyter_output><empty_output><jupyter_text>Download Data for Studying- Download iRobot historical stock prices for the last 10 years (2008-2018) from yahoo financial- Save downloaded data into a csv file and store locally<jupyter_code>import pandas_datareader as pdr

financial_symbol = "IRBT"

def load_stock_data():
    # Save data to this file
    file_to_save = 'data/stock_data-%s.csv'%financial_symbol

    # Try to find the local data first -- if doesn't exist, download the new one
    if not os.path.exists(file_to_save):
        # Get the financial data with symbol "IRBT" using data reader
        data = pdr.get_data_yahoo(symbols=financial_symbol, start=datetime(1998, 1, 1), end=datetime(2018, 1, 1))
        data.to_csv(file_to_save)
        print('Data saved to : %s'%file_to_save)
    else:
        print('File already exists. Loading data from CSV')
        
    data = pd.read_csv(file_to_save)
    return data

# Sort the values to make sure we have sorted list by date increasingly
irbt = load_stock_data()
irbt.sort_values('Date')
irbt.head()<jupyter_output>File already exists. Loading data from CSV<jupyter_text>Visualizing Data- Describing what the data looks like- Display all the data as a graph to see how the stock prices looks like overall for the last 10 years<jupyter_code># Describing data set
irbt.describe()
# Infomation of the data set
irbt.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 2519 entries, 0 to 2518
Data columns (total 7 columns):
Date         2519 non-null object
High         2519 non-null float64
Low          2519 non-null float64
Open         2519 non-null float64
Close        2519 non-null float64
Volume       2519 non-null int64
Adj Close    2519 non-null float64
dtypes: float64(5), int64(1), object(1)
memory usage: 137.8+ KB<jupyter_text>Looking at the information alone, we can see that there are 2519 entries total and there is 2519 entries for each and every entries. Thus, it's safe to say that we have a good data set since no entry is missing.<jupyter_code>import matplotlib.pyplot as plt

plt.figure(figsize=(18,9))
plt.plot(range(irbt.shape[0]),(irbt['Low'] + irbt['High'])/2.0)
plt.xlabel('Date',fontsize=18)
plt.ylabel('Mid Price (USD)',fontsize=18)

# Sticks for every 3 months
plt.xticks(range(0,irbt.shape[0],90), irbt['Date'].loc[::90], rotation=45)
plt.show()<jupyter_output><empty_output><jupyter_text>Splitting Data into Training Set and Test Set- Taking about 20% amount of data to be the test data, the rest will be used as train data.- Also, we want to avoid making the ML algorithms gets to see the whole data set. Thus, we need to randomly load a subsequence of 80% amount of data<jupyter_code>from sklearn.model_selection import train_test_split

# Convert to numpy array
irbt = irbt.values

# Training and test data
train_start = 0
train_end = int(np.floor(0.8*irbt.shape[0]))
test_start = train_end
test_end = irbt.shape[0]

train_set = irbt[np.arange(train_start, train_end), 1:]
test_set = irbt[np.arange(test_start, test_end), 1:]<jupyter_output><empty_output><jupyter_text>Scale data- Transform training set and test set from 1D array to 2D array- Scale up training data and testing data in range of [-1, 1]<jupyter_code>from sklearn.preprocessing import MinMaxScaler

# Scale using a min max scaler -- since most of stock data are positive, so range should be [0,1]
scaler = MinMaxScaler()
train_data = scaler.fit_transform(train_set)
test_data = scaler.transform(test_set)

# Build X and y
X_train = train_data[1:, 1:]
y_train = train_data[1:, 0]
X_test = test_data[1:, 1:]
y_test = test_data[1:, 0]

print(y_train)
print(X_train)<jupyter_output>[0.26053079 0.25273922 0.25054784 ... 0.71682491 0.71487707 0.69905039]
[[0.27785128 0.27942311 0.27497398 0.02205708 0.27497398]
 [0.25826938 0.2704095  0.25780437 0.04770419 0.25780437]
 [0.26620797 0.26036569 0.26742973 0.093327   0.26742973]
 ...
 [0.76395867 0.75560138 0.76092606 0.03710171 0.76092606]
 [0.76766334 0.753026   0.75286161 0.02152625 0.75286161]
 [0.75125691 0.74169459 0.7362123  0.02318857 0.7362123 ]]<jupyter_text>Tensorflow Integration Declare tensorflow placeholders for our X and y training data- We now know that the stock values are floats in term of US Dollars, so place holder type is necessary a float 32- However, we don't know just yet the amount of value flowing into this place holder, so make sure don't give specific type yet<jupyter_code># Placeholder
X = tf.placeholder(dtype=tf.float32, shape=[None, 5])
Y = tf.placeholder(dtype=tf.float32, shape=[None])<jupyter_output><empty_output><jupyter_text>Design Neural NetworksThe design that we are using is following image below. We will have a layer of input data, 4 hidden layers and 1 layout of output data![neural_networks.png](attachment:neural_networks.png) Create of Weight and Biases<jupyter_code># Model architecture parameters
num_stocks = 5
num_neurons_layer_1 = 1024
num_neurons_layer_2 = 512
num_neurons_layer_3 = 256
num_neurons_layer_4 = 128
num_neurons_target = 1

# Initializers
sigma = 1
weight_initializer = tf.variance_scaling_initializer(mode="fan_avg", distribution="uniform", scale=sigma)
bias_initializer = tf.zeros_initializer()

# Layer 1: Variables for hidden weights and biases
weight_hidden_1 = tf.Variable(weight_initializer([num_stocks, num_neurons_layer_1]))
bias_hidden_1 = tf.Variable(bias_initializer([num_neurons_layer_1]))

# Layer 2: Variables for hidden weights and biases
weight_hidden_2 = tf.Variable(weight_initializer([num_neurons_layer_1, num_neurons_layer_2]))
bias_hidden_2 = tf.Variable(bias_initializer([num_neurons_layer_2]))

# Layer 3: Variables for hidden weights and biases
weight_hidden_3 = tf.Variable(weight_initializer([num_neurons_layer_2, num_neurons_layer_3]))
bias_hidden_3 = tf.Variable(bias_initializer([num_neurons_layer_3]))

# Layer 4: Variables for hidden weights and biases
weight_hidden_4 = tf.Variable(weight_initializer([num_neurons_layer_3, num_neurons_layer_4]))
bias_hidden_4 = tf.Variable(bias_initializer([num_neurons_layer_4]))

# Output layer: Variables for output weights and biases
weight_out = tf.Variable(weight_initializer([num_neurons_layer_4, num_neurons_target]))
bias_out = tf.Variable(bias_initializer([num_neurons_target]))<jupyter_output><empty_output><jupyter_text>Neutral Networks Architecture Implementation- Since we already have weights and biases, we need to combine them together into a system of matrix multiplications<jupyter_code># Hidden layer -- Refer to image above to understand more about the neutral layer
hidden_layer_1 = tf.nn.relu(tf.add(tf.matmul(X, weight_hidden_1), bias_hidden_1))
hidden_layer_2 = tf.nn.relu(tf.add(tf.matmul(hidden_layer_1, weight_hidden_2), bias_hidden_2))
hidden_layer_3 = tf.nn.relu(tf.add(tf.matmul(hidden_layer_2, weight_hidden_3), bias_hidden_3))
hidden_layer_4 = tf.nn.relu(tf.add(tf.matmul(hidden_layer_3, weight_hidden_4), bias_hidden_4))

# Output layer (must be transposed)
output_layer = tf.transpose(tf.add(tf.matmul(hidden_layer_4, weight_out), bias_out))<jupyter_output><empty_output><jupyter_text>Cost function- Create a cost function to measure deviation between the network’s predictions and the actual observed training targets- For regression problems, the mean squared error (MSE) function is commonly used. MSE computes the average squared deviation between predictions and targets<jupyter_code>mse_func = tf.reduce_mean(tf.squared_difference(output_layer, Y))<jupyter_output><empty_output><jupyter_text>Create optimizer- User a optimizer to make sure that the value of weights and biases will change in the correct way- We will be using Adam Optimizer which is one of the current default optimizers in deep learning development.<jupyter_code># Optimizer
opt = tf.train.AdamOptimizer().minimize(mse_func)<jupyter_output><empty_output><jupyter_text>Fitting the neural network- Train our model: After having defined the placeholders, variables, initializers, cost functions and optimizers of the network, we can start training our model.<jupyter_code># Make Session
net = tf.Session()

# Run initializer
net.run(tf.global_variables_initializer())

# Setup interactive plot
plt.ion()
fig = plt.figure()
ax1 = fig.add_subplot(111)
line1, = ax1.plot(y_test)
line2, = ax1.plot(y_test*0.5)
plt.show()

# Number of epochs and batch size
epochs = 20
batch_size = 512

for e in range(epochs):
    # Shuffle training data
    shuffle_indices = np.random.permutation(np.arange(len(y_train)))
    X_train = X_train[shuffle_indices]
    y_train = y_train[shuffle_indices]
    
    # Minibatch training
    for i in range(0, len(y_train) // batch_size):
        start = i * batch_size
        batch_x = X_train[start:start + batch_size]
        batch_y = y_train[start:start + batch_size]
        
        # Run optimizer with batch
        net.run(opt, feed_dict={X: batch_x, Y: batch_y})

        # Show progress every 5 batches
        if np.mod(i, 2) == 0:
            # Prediction
            pred = net.run(output_layer, feed_dict={X: X_test})
            line2.set_ydata(pred)
            
# Print final MSE after Training
mse_final = net.run(mse_func, feed_dict={X: X_test, Y: y_test})
print(mse_final)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08933080808080808, 'fraction_numerical': 0.057133838383838384, 'mean_word_length': 5.208360548661006, 'pattern_counts': {'":': 0, '<': 47, '<?xml version=': 0, '>': 47, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '139504', 'license': 'Apache-2.0', 'n_tokens_mistral': 3500, 'n_tokens_neox': 3118, 'n_words': 940, 'path': 'stock study/iRobot Stock Study.ipynb', 'repo_name': 'hnguyenworkstation/iRobot-Stock-Prediction', 'size': '9504'}