<jupyter_start><jupyter_text>Neural Networks for Time Series**Author:** <NAME>Project for Econometrics III (2018) class.We compare traditional time series methods like ARIMA models and novel approaches like Neural Networks.  LSTM neural networks<jupyter_code>from supervised_ts import series_to_supervised
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
from keras.models import Sequential
from keras.layers.core import Dense, Activation, Dropout
from keras.layers.recurrent import LSTM
from sklearn.preprocessing import MinMaxScaler
from sklearn.metrics import mean_squared_error

# Configure visualizations
%config InlineBackend.figure_format = 'retina'
%matplotlib inline
plt.rcParams["figure.figsize"] = [7,4]

# Ignore warnings
import warnings
warnings.filterwarnings('ignore')
print('-'*25)<jupyter_output>-------------------------<jupyter_text>Quito Store  Part 0: Load & Preprocess dataThe data consists of retail stores' transactions. The stores are from Quito (44) and from Guayaquil (34).<jupyter_code># Fix random seed for reproducibility
np.random.seed(42)

# Store to predict transactions
store = 44<jupyter_output><empty_output><jupyter_text>We have to normalize the data before put it in the neural network. We are using 2016 and above.<jupyter_code># Load the dataset
df_transactions = pd.read_csv(r'..\..\data\input\transaction{}.csv'.format(store), usecols=[0, 2], index_col=0)

# Using 2016 and above
df_transactions.index = df_transactions.index.to_datetime()
df_transactions = df_transactions[df_transactions.index.year>=2016]

# Normalize the dataset
scaler = MinMaxScaler(feature_range=(0, 1))
scaled_values = scaler.fit_transform(df_transactions)
df_transactions["transactions"] = scaled_values

# Split into train and test sets
y = df_transactions.astype(float)
train_size = int(len(y) * 0.70)
train, test = y[0:train_size], y[train_size:]
print("train: {}, test: {}".format(len(train), len(test)))<jupyter_output>train: 411, test: 177<jupyter_text>I have to define the max value of lags I need. The function series_to_supervised helps me lagging my dependent variable. Finally I need to define train-test data.<jupyter_code>max_lags = 15
train = series_to_supervised(train, max_lags, 1)
test = series_to_supervised(test, max_lags, 1)

output_list = ["var1(t)"]
xtrain, ytrain = train.iloc[:,:-1], train.loc[:,output_list]
xtest, ytest = test.iloc[:,:-1], test.loc[:,output_list]<jupyter_output><empty_output><jupyter_text>The LSTM network expects the input data (X) to be provided with a specific array structure in the form of: [samples, time steps, features].Currently, my data is in the form: [samples, features] and we are framing the problem as one time step for each sample. We can transform the prepared train and test input data into the expected structure using numpy.reshape() as follows:<jupyter_code># reshape input to be [samples, time steps, features]
xtrain = xtrain.values
xtest = xtest.values
ytrain = ytrain.values
ytest = ytest.values
xtrain = np.reshape(xtrain, (xtrain.shape[0], 1, xtrain.shape[1]))
xtest = np.reshape(xtest, (xtest.shape[0], 1, xtest.shape[1]))<jupyter_output><empty_output><jupyter_text>Part 1: Design Network The network has a visible layer with 1 input, a hidden layer with 50 LSTM blocks or neurons, and an output layer that makes a single value prediction. The default sigmoid activation function is used for the LSTM blocks. The network is trained for 50 epochs and a batch size of 1 is used.<jupyter_code># design network
model = Sequential()
model.add(LSTM(50, input_shape=(xtrain.shape[1], xtrain.shape[2])))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')
# fit network
history = model.fit(xtrain, ytrain, epochs=50, batch_size=1, validation_data=(xtest, ytest), verbose=0, shuffle=False)
history.history.keys()<jupyter_output><empty_output><jupyter_text>It seems that the loss value converge.<jupyter_code># plot history
# History is a dictionary with dict_keys(['val_loss', 'loss'])
plt.plot(history.history['loss'], label='train')
plt.plot(history.history['val_loss'], label='test')
plt.legend()
plt.show()<jupyter_output><empty_output><jupyter_text>Part 2: Forecasting LSTM I have to re-scale the values to make comparisons.<jupyter_code>yhat = model.predict(xtest)
inv_yhat = scaler.inverse_transform(yhat)
inv_y = scaler.inverse_transform(y)

y_pred = pd.Series(inv_yhat.reshape(-1), index=df_transactions[train_size+max_lags:].index)
y = pd.Series(inv_y.reshape(-1), index=df_transactions.index)

rmse = np.sqrt(mean_squared_error(y[train_size+max_lags:], y_pred))
print('Test RMSE: %.2f' % rmse)
ax = y.plot(label='observed', figsize=(16, 5), color="darkred")
y_pred.plot(ax=ax, label='forecast', color="forestgreen")

plt.title("Forecasting transactions of Quito store", fontsize=20)
plt.legend()
# plt.savefig('../figures/forecast_lstm{}.png'.format(store), dpi=300, bbox_inches='tight')
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12057165861513687, 'fraction_numerical': 0.01710950080515298, 'mean_word_length': 6.597859327217125, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '142374', 'license': 'MIT', 'n_tokens_mistral': 1711, 'n_tokens_neox': 1614, 'n_words': 462, 'path': 'Python_Code/Neural_Networks_Time_Series.ipynb', 'repo_name': 'pabloestradac/Time_Series_Notebooks', 'size': '4975'}