<jupyter_start><jupyter_text>RNN on Time series datasetOn this Notebook we will build a recurrent model on a multi-variate time series dataset. The dataThe dataset is a multi-variate time series. Let's do a very short EDA on it:<jupyter_code>import pandas as pd
from datetime import datetime
import matplotlib.pyplot as plt
# load data
def parse(x):
    return datetime.strptime(x, '%Y %m %d %H')
dataset_path = '/home/fer/data/formaciones/master/deep-learning-intro/datasets/time_series/pollution.csv'
dataset = pd.read_csv(dataset_path,  parse_dates = [['year', 'month', 'day', 'hour']], index_col=0, date_parser=parse)
dataset.drop('No', axis=1, inplace=True)


values = dataset.values
# specify columns to plot
groups = [0, 1, 2, 3, 5, 6, 7]
i = 1
# plot each column
plt.figure(figsize=(10,10))
for group in groups:
    plt.subplot(len(groups), 1, i)
    plt.plot(values[:, group])
    plt.title(dataset.columns[group], y=0.5, loc='right')
    i += 1
plt.show()

# manually specify column names
dataset.columns = ['pollution', 'dew', 'temp', 'press', 'wnd_dir', 'wnd_spd', 'snow', 'rain']
dataset.index.name = 'date'

# mark all NA values with 0
dataset['pollution'].fillna(0, inplace=True)
# drop the first 24 hours
dataset = dataset[24:]
# summarize first 5 rows
dataset.head()<jupyter_output><empty_output><jupyter_text>This is what we have. It should be necessary to spend more time on reviewing it, but we will focus on the RNN application. Let's first get rid of the categorical feature by using a label encoder (not the best solution for sure) and scale the now continuous features.Scale the datasetHint: use the minmaxscaler function<jupyter_code>from sklearn.preprocessing import MinMaxScaler
from sklearn.preprocessing import LabelEncoder

values = dataset.values
# integer encode direction
encoder = LabelEncoder()
values[:,4] = encoder.fit_transform(values[:,4])
values = values.astype('float32')
...<jupyter_output><empty_output><jupyter_text>The next function converts our series data into a supervised learning one, by shifting it and adding timesteps to each row:<jupyter_code># convert series to supervised learning
def series_to_supervised(data, n_in=1, n_out=1, dropnan=True):
    """
    Frame a time series as a supervised learning dataset.
    Arguments:
        data: Sequence of observations as a list or NumPy array.
        n_in: Number of lag observations as input (X).
        n_out: Number of observations as output (y).
        dropnan: Boolean whether or not to drop rows with NaN values.
    Returns:
        Pandas DataFrame of series framed for supervised learning.
    """
    n_vars = data.shape[1]
    df = pd.DataFrame(data)
    cols, names = list(), list()
    # input sequence (t-n, ... t-1)
    for i in range(n_in, 0, -1):
        cols.append(df.shift(i))
        names += [('var%d(t-%d)' % (j+1, i)) for j in range(n_vars)]
    # forecast sequence (t, t+1, ... t+n)
    for i in range(0, n_out):
        cols.append(df.shift(-i))
        if i == 0:
            names += [('var%d(t)' % (j+1)) for j in range(n_vars)]
        else:
            names += [('var%d(t+%d)' % (j+1, i)) for j in range(n_vars)]
    # put it all together
    agg = pd.concat(cols, axis=1)
    agg.columns = names
    # drop rows with NaN values
    if dropnan:
        agg.dropna(inplace=True)
    return agg



# frame as supervised learning
reframed = series_to_supervised(scaled, 1, 1)

reframed.head()<jupyter_output><empty_output><jupyter_text>This way, in our example, we have the feature values for the current timestep and the previous. The idea is to predict the next from the current. So we are interested in predicting $var1(t)$ from $var1(t-1), var2(t-1), \cdots, var8(t-1)$. Let's drop the useless features:<jupyter_code># drop columns we don't want to predict
reframed.drop(reframed.columns[[9,10,11,12,13,14,15]], axis=1, inplace=True)
reframed.head()<jupyter_output><empty_output><jupyter_text>Let's divide it now in traiing and testing. In order to do that, we can leave the last year as testing.Split the data<jupyter_code># split into train and test sets
values = reframed.values
...
train = ...
test = ...
# split into input and outputs
train_X, train_y = train[:, :-1], train[:, -1]
test_X, test_y = test[:, :-1], test[:, -1]
# reshape input to be 3D [samples, timesteps, features]
train_X = train_X.reshape((train_X.shape[0], 1, train_X.shape[1]))
test_X = test_X.reshape((test_X.shape[0], 1, test_X.shape[1]))
print(train_X.shape, train_y.shape, test_X.shape, test_y.shape)<jupyter_output><empty_output><jupyter_text>Please note the dataset shape: (N_samples, timesteps, n_features).  Model Architecture:We will build now an LSTM network in order to predict the target.- Use a LSTM layer, with 50 units.- Use a dense layer with a single feature: the one we are trying to predict.- Compile it using mae as the loss and adam as the optimizer.- train it using the test data as validation, not shuffling the data (why?) and using 10 epochs and a batch size of 72.- Store the fitting inside a history variable, for plotting purposes.Build the model<jupyter_code>import tensorflow as tf
physical_devices = tf.config.experimental.list_physical_devices('GPU')
tf.config.experimental.set_memory_growth(physical_devices[0], True)
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import LSTM
# design network
...
# fit network
history = ...<jupyter_output><empty_output><jupyter_text>Now let's plot the loss. We have a few epochs, but we can see the prgress:<jupyter_code># plot history
plt.plot(history.history['loss'], label='train')
plt.plot(history.history['val_loss'], label='test')
plt.legend()
plt.show()<jupyter_output><empty_output><jupyter_text>Finally, let's see how well we did. Keep in mind that the model does predict at scaled level. So we need to de-escalate:<jupyter_code>import numpy as np
# make a prediction
yhat = model.predict(test_X)
x_test = test_X.reshape((test_X.shape[0], test_X.shape[2]))
# invert scaling for forecast
inv_yhat = np.concatenate((yhat, x_test[:, 1:]), axis=1)
inv_yhat = scaler.inverse_transform(inv_yhat)
inv_yhat = inv_yhat[:,0]
# invert scaling for actual
y_test = test_y.reshape((len(test_y), 1))
inv_y = np.concatenate((y_test, x_test[:, 1:]), axis=1)
inv_y = scaler.inverse_transform(inv_y)
inv_y = inv_y[:,0]<jupyter_output><empty_output><jupyter_text>Compute the rmse and mae at test level<jupyter_code>from sklearn.metrics import ...
from math import sqrt
# calculate RMSE
rmse = ... 
mae = ...
print('Test RMSE: %.3f' % rmse)
print('Test MAE: %.3f' % mae)<jupyter_output><empty_output><jupyter_text>And finally, let's plot the first 100 results:<jupyter_code>plt.figure(figsize=(10,10))
plt.plot(inv_y[0:100], marker='.', label="true")
plt.plot(inv_yhat[0:100], 'r', label="prediction")
plt.ylabel('Value')
plt.xlabel('Time Step')
plt.legend()
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1244956772334294, 'fraction_numerical': 0.01570605187319885, 'mean_word_length': 4.680032733224222, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 41, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '146243', 'license': 'MIT', 'n_tokens_mistral': 2427, 'n_tokens_neox': 2309, 'n_words': 749, 'path': 'training/TimeSeriesTraining.ipynb', 'repo_name': 'serCliff/deep-learning-intro', 'size': '6940'}