<jupyter_start><jupyter_text>Data cleansing<jupyter_code>columns_to_keep = ['Passengers']
df = df[columns_to_keep]
df['Passengers'] = df['Passengers'].apply(lambda x: x*1000)
df.index.names = ['Month']
df.sort_index(inplace=True)
print('Total rows: {}'.format(len(df)))
df.head()
df.describe()
df.plot()<jupyter_output><empty_output><jupyter_text>Null values?<jupyter_code>df.isnull().sum()
null_columns=df.columns[df.isnull().any()]
df[null_columns].isnull().sum()
print(df[df.isnull().any(axis=1)][null_columns].head())
df.dropna(inplace=True)
df.isnull().sum()
df.hist(bins=10)
len(df[df['Passengers'] == 0])<jupyter_output><empty_output><jupyter_text>Scaled data<jupyter_code>print('Min', np.min(df))
print('Max', np.max(df))<jupyter_output>Min Passengers    104000.0
dtype: float64
Max Passengers    622000.0
dtype: float64<jupyter_text>We can then extract the NumPy array from the dataframe and convert the integer values to floating point values, which are more suitable for modeling with a neural network.<jupyter_code>dataset = df.astype('float32')<jupyter_output><empty_output><jupyter_text>LSTMs are sensitive to the scale of the input data, specifically when the sigmoid (default) or tanh activation functions are used. It can be a good practice to rescale the data to the range of 0-to-1, also called normalizing. We can easily normalize the dataset using the MinMaxScaler preprocessing class from the scikit-learn library.<jupyter_code>scaler = MinMaxScaler(feature_range=(0, 1))
scaled = scaler.fit_transform(dataset) 
print('Min', np.min(scaled))
print('Max', np.max(scaled))
print(scaled[:10])<jupyter_output>[[0.01544401]
 [0.02702703]
 [0.05405405]
 [0.04826255]
 [0.03281853]
 [0.05984557]
 [0.08494207]
 [0.08494207]
 [0.06177607]
 [0.02895753]]<jupyter_text>Create the RNN A simple method that we can use is to split the ordered dataset into train and test datasets. The code below calculates the index of the split point and separates the data into the training datasets with 70% of the observations that we can use to train our model, leaving the remaining 30% for testing the model.Split into train and test sets<jupyter_code>train_size = int(len(scaled) * 0.70)
test_size = len(scaled - train_size)
train, test = scaled[0:train_size, :], scaled[train_size: len(scaled), :]
print('train: {}\ntest: {}'.format(len(train), len(test)))<jupyter_output>train: 100
test: 44<jupyter_text>convert an array of values into a dataset matrix The function takes two arguments: the dataset, which is a NumPy array that we want to convert into a dataset, and the look_back, which is the number of previous time steps to use as input variables to predict the next time period — in this case defaulted to 1. This default will create a dataset where X is the energy quantity at a given time (t) and Y is the qty of energy at the next time (t + 1).<jupyter_code>def create_dataset(dataset, look_back=1):
    print(len(dataset), look_back)
    dataX, dataY = [], []
    for i in range(len(dataset)-look_back-1):
        a = dataset[i:(i+look_back), 0]
        print(i)
        print('X {} to {}'.format(i, i+look_back))
        print(a)
        print('Y {}'.format(i + look_back))
        print(dataset[i + look_back, 0])
        dataset[i + look_back, 0]
        dataX.append(a)
        dataY.append(dataset[i + look_back, 0])
    return np.array(dataX), np.array(dataY)<jupyter_output><empty_output><jupyter_text>reshape into X=t and Y=t+1<jupyter_code>look_back = 1
X_train, y_train = create_dataset(train, look_back)
X_test, y_test = create_dataset(test, look_back)<jupyter_output>100 1
0
X 0 to 1
[0.01544401]
Y 1
0.027027026
1
X 1 to 2
[0.02702703]
Y 2
0.05405405
2
X 2 to 3
[0.05405405]
Y 3
0.04826255
3
X 3 to 4
[0.04826255]
Y 4
0.032818526
4
X 4 to 5
[0.03281853]
Y 5
0.059845567
5
X 5 to 6
[0.05984557]
Y 6
0.08494207
6
X 6 to 7
[0.08494207]
Y 7
0.08494207
7
X 7 to 8
[0.08494207]
Y 8
0.06177607
8
X 8 to 9
[0.06177607]
Y 9
0.02895753
9
X 9 to 10
[0.02895753]
Y 10
0.0
10
X 10 to 11
[0.]
Y 11
0.027027026
11
X 11 to 12
[0.02702703]
Y 12
0.021235526
12
X 12 to 13
[0.02123553]
Y 13
0.042471036
13
X 13 to 14
[0.04247104]
Y 14
0.07142857
14
X 14 to 15
[0.07142857]
Y 15
0.059845567
15
X 15 to 16
[0.05984557]
Y 16
0.04054053
16
X 16 to 17
[0.04054053]
Y 17
0.08687258
17
X 17 to 18
[0.08687258]
Y 18
0.12741312
18
X 18 to 19
[0.12741312]
Y 19
0.12741312
19
X 19 to 20
[0.12741312]
Y 20
0.10424709
20
X 20 to 21
[0.10424709]
Y 21
0.055984557
21
X 21 to 22
[0.05598456]
Y 22
0.01930502
22
X 22 to 23
[0.01930502]
Y 23
0.06949806
23
X 23 to 24
[0.06949806]
Y 24
0.07915059
24
X 24[...]<jupyter_text>The LSTM network expects the input data (X) to be provided with a specific array structure in the form of: [samples, time steps, features].reshape input to be [samples, time steps, features]<jupyter_code>X_train = np.reshape(X_train, (X_train.shape[0], X_train.shape[1], 1))
X_test = np.reshape(X_test, (X_test.shape[0], X_test.shape[1], 1))
print(X_train.shape)
print(X_test.shape)<jupyter_output>(98, 1, 1)
(42, 1, 1)<jupyter_text>The network has a visible layer with 1 input, a hidden layer with 4 LSTM blocks or neurons, and an output layer that makes a single value prediction. The default sigmoid activation function is used for the LSTM blocks. The network is trained for 100 epochs and a batch size of 1 is used.create and fit the LSTM network<jupyter_code>batch_size = 1
model = Sequential()
model.add(LSTM(4, batch_input_shape=(batch_size, look_back, 1), stateful=True))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')
model.fit(X_train, y_train, epochs=100, batch_size=batch_size, verbose=2, shuffle=True)<jupyter_output>Epoch 1/100
98/98 - 1s - loss: 0.0209
Epoch 2/100
98/98 - 0s - loss: 0.0163
Epoch 3/100
98/98 - 0s - loss: 0.0147
Epoch 4/100
98/98 - 0s - loss: 0.0147
Epoch 5/100
98/98 - 0s - loss: 0.0138
Epoch 6/100
98/98 - 0s - loss: 0.0118
Epoch 7/100
98/98 - 0s - loss: 0.0115
Epoch 8/100
98/98 - 0s - loss: 0.0106
Epoch 9/100
98/98 - 0s - loss: 0.0083
Epoch 10/100
98/98 - 0s - loss: 0.0080
Epoch 11/100
98/98 - 0s - loss: 0.0064
Epoch 12/100
98/98 - 0s - loss: 0.0067
Epoch 13/100
98/98 - 0s - loss: 0.0049
Epoch 14/100
98/98 - 0s - loss: 0.0045
Epoch 15/100
98/98 - 0s - loss: 0.0033
Epoch 16/100
98/98 - 0s - loss: 0.0031
Epoch 17/100
98/98 - 0s - loss: 0.0024
Epoch 18/100
98/98 - 0s - loss: 0.0022
Epoch 19/100
98/98 - 0s - loss: 0.0023
Epoch 20/100
98/98 - 0s - loss: 0.0022
Epoch 21/100
98/98 - 0s - loss: 0.0021
Epoch 22/100
98/98 - 0s - loss: 0.0022
Epoch 23/100
98/98 - 0s - loss: 0.0023
Epoch 24/100
98/98 - 0s - loss: 0.0022
Epoch 25/100
98/98 - 0s - loss: 0.0021
Epoch 26/100
98/98 - 0s - loss: 0.[...]<jupyter_text>Note that we invert the predictions before calculating error scores to ensure that performance is reported in the same units as the original data (thousands of passengers per month).Make preditions<jupyter_code>import math
from sklearn.metrics import mean_squared_error

trainPredict = model.predict(X_train, batch_size=batch_size)
model.reset_states()

testPredict = model.predict(X_test, batch_size=batch_size)
# invert predictions
trainPredict = scaler.inverse_transform(trainPredict)
y_train = scaler.inverse_transform([y_train])
testPredict = scaler.inverse_transform(testPredict)
y_test = scaler.inverse_transform([y_test])
# calculate root mean squared error
trainScore = math.sqrt(mean_squared_error(y_train[0], trainPredict[:,0]))
print('Train Score: %.2f RMSE' % (trainScore))
testScore = math.sqrt(mean_squared_error(y_test[0], testPredict[:,0]))
print('Test Score: %.2f RMSE' % (testScore))<jupyter_output>Train Score: 23467.20 RMSE
Test Score: 50436.33 RMSE<jupyter_text>Because of how the dataset was prepared, we must shift the predictions so that they align on the x-axis with the original dataset. Once prepared, the data is plotted, showing the original dataset in blue, the predictions for the training dataset in green, and the predictions on the unseen test dataset in red.shift train predictions for plotting<jupyter_code>trainPredictPlot = np.empty_like(scaled)
trainPredictPlot[:, :] = np.nan
trainPredictPlot[look_back:len(trainPredict)+look_back, :] = trainPredict
# shift test predictions for plotting
testPredictPlot = np.empty_like(scaled)
testPredictPlot[:, :] = np.nan
testPredictPlot[len(trainPredict)+(look_back*2)+1:len(scaled)-1, :] = testPredict
# plot baseline and predictions
plt.figure(figsize=(20,10))
plt.plot(scaler.inverse_transform(scaled))
plt.plot(trainPredictPlot)
plt.plot(testPredictPlot)
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11739992998016105, 'fraction_numerical': 0.13607188703465983, 'mean_word_length': 4.645586297760211, 'pattern_counts': {'":': 0, '<': 42, '<?xml version=': 0, '>': 42, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '13304', 'license': 'MIT', 'n_tokens_mistral': 4055, 'n_tokens_neox': 3260, 'n_words': 1012, 'path': 'timeseries-prediction.ipynb', 'repo_name': 'pdrok/timeseries-lstm-keras', 'size': '8569'}