<jupyter_start><jupyter_text>Dataset<jupyter_code>df = pd.read_csv(
    'cansim-0800020-eng-6674700030567901031.csv',
    skiprows=6, skipfooter=9, engine='python')
df.head()<jupyter_output><empty_output><jupyter_text>Adjust date<jupyter_code>from pandas.tseries.offsets import MonthEnd

df['Adjustments'] = pd.to_datetime(df['Adjustments']) + MonthEnd(1)
df = df.set_index('Adjustments')

display(df.head())
display(df.plot())<jupyter_output><empty_output><jupyter_text>Split train-testset<jupyter_code>split_date = pd.Timestamp('01-01-2011')

train = df.loc[:split_date, ['Unadjusted']]
test = df.loc[split_date:, ['Unadjusted']]
ax = train.plot();
test.plot(ax=ax)
plt.legend(['train', 'test']);<jupyter_output><empty_output><jupyter_text>MinMax Scaling input<jupyter_code>from sklearn.preprocessing import MinMaxScaler

sc = MinMaxScaler()

train_sc = sc.fit_transform(train)
test_sc = sc.transform(test)
ax = pd.DataFrame(train_sc).plot();
pd.DataFrame(test_sc).plot(ax=ax)
plt.legend(['train', 'test']);<jupyter_output><empty_output><jupyter_text>Create Sliding Windows<jupyter_code>train_sc_df = pd.DataFrame(train_sc, columns=['MinMax_Scaled'], index=train.index)
test_sc_df = pd.DataFrame(test_sc, columns=['MinMax_Scaled'], index=test.index)

train_sc_df.head()
for s in range(1, 13):
    train_sc_df['shift_{}'.format(s)] = train_sc_df['MinMax_Scaled'].shift(s)
    test_sc_df['shift_{}'.format(s)] = test_sc_df['MinMax_Scaled'].shift(s)
    
train_sc_df.head(13)<jupyter_output><empty_output><jupyter_text>Define train-testset<jupyter_code>X_train = train_sc_df.dropna().drop('MinMax_Scaled', axis=1)
y_train = train_sc_df.dropna()[['MinMax_Scaled']]

X_test = test_sc_df.dropna().drop('MinMax_Scaled', axis=1)
y_test = test_sc_df.dropna()[['MinMax_Scaled']]

display(X_train.head())
display(y_train.head())
X_train = X_train.values
X_test= X_test.values

y_train = y_train.values
y_test = y_test.values

print(X_train.shape)
print(y_train.shape)
print('')
print(X_test.shape)
print(y_test.shape)<jupyter_output>(228, 12)
(228, 1)

(61, 12)
(61, 1)<jupyter_text>Exercise 1In the model above we reshaped the input shape to: `(num_samples, 1, 12)`, i.e. we treated a window of 12 months as a vector of 12 coordinates that we simultaneously passed to all the LSTM nodes. An alternative way to look at the problem is to reshape the input to `(num_samples, 12, 1)`. This means we consider each input window as a sequence of 12 values that we will pass in sequence to the LSTM. In principle this looks like a more accurate description of our situation. But does it yield better predictions? Let's check it.- Reshape `X_train` and `X_test` so that they represent a set of univariate sequences- retrain the same LSTM(6) model, you'll have to adapt the `input_shape`- check the performance of this new model, is it better at predicting the test data?<jupyter_code># Convert to tensor: (num_samples, 12, 1)
X_train_t = X_train.reshape(X_train.shape[0], 12, 1)
X_test_t = X_test.reshape(X_test.shape[0], 12, 1)

print(X_train_t.shape)
print(X_test_t.shape)
from keras.models import Sequential
import keras.backend as K
from keras.layers import LSTM, Dense
from keras.callbacks import EarlyStopping

K.clear_session()

model = Sequential()

model.add(LSTM(6, input_shape=(12, 1)))
model.add(Dense(1))

model.compile(loss='mean_squared_error', optimizer='adam')
model.summary()
early_stop = EarlyStopping(monitor='loss', patience=3, verbose=1)
history = model.fit(
    X_train_t, y_train,
    epochs=100, batch_size=1, verbose=0,
    validation_split=0.25,
    callbacks=[early_stop])
historydf = pd.DataFrame(history.history, index=history.epoch)
historydf.plot();<jupyter_output><empty_output><jupyter_text>Evaluate<jupyter_code>y_train_pred = model.predict(X_train_t)
y_test_pred = model.predict(X_test_t)
from sklearn.metrics import mean_squared_error as mse

print("The Mean Squared Error on the Train set is:\t{:0.5f}".format(mse(y_train, y_train_pred)))
print("The Mean Squared Error on the Test set is:\t{:0.5f}".format(mse(y_test, y_test_pred)))
from sklearn.metrics import r2_score

print("The R2 score on the Train set is:\t{:0.3f}".format(r2_score(y_train, y_train_pred)))
print("The R2 score on the Test set is:\t{:0.3f}".format(r2_score(y_test, y_test_pred)))
plt.plot(y_test)
plt.plot(y_test_pred)
plt.legend(['real vals', 'predicted']);<jupyter_output><empty_output><jupyter_text>=> The models are extremely good  Exercise 2RNN models can be applied to images too. In general we can apply them to any data where there's a connnection between nearby units. Let's see how we can easily build a model that works with images.- Load the MNIST data, by now you should be able to do it blindfolded :)- reshape it so that an image looks like a long sequence of pixels- create a recurrent model and train it on the training data- how does it perform compared to a fully connected? How does it compare to Convolutional Neural Networks?(feel free to run this exercise on a cloud GPU if it's too slow on your laptop)  Load dataset<jupyter_code>from keras.datasets import mnist

(X_train, y_train), (X_test, y_test) = mnist.load_data('/tmp/mnist.npz')


print(X_train.shape)
print(y_train.shape)
print('')
print(X_test.shape)
print(y_test.shape)
plt.imshow(X_train[0], cmap='gray');
print(y_train[0])<jupyter_output>5<jupyter_text>Feature Engineering X<jupyter_code># Normalize X
X_train = X_train.astype('float32') / 255.0
X_test = X_test.astype('float32') / 255.0

# Convert to tensor
X_train = X_train.reshape(X_train.shape[0], -1, 1)
X_test = X_test.reshape(X_test.shape[0], -1, 1)

print(X_train.shape)
print(X_test.shape)<jupyter_output>(60000, 784, 1)
(10000, 784, 1)<jupyter_text>One hot encoding y<jupyter_code>from keras.utils.np_utils import to_categorical

y_train_cat = to_categorical(y_train)
y_test_cat = to_categorical(y_test)

print(y_train_cat.shape)
print(y_test_cat.shape)<jupyter_output>(60000, 10)
(10000, 10)<jupyter_text>Model<jupyter_code>from keras.models import Sequential
import keras.backend as K
from keras.layers import LSTM, Dense
from keras.callbacks import EarlyStopping

K.clear_session()

model = Sequential()

model.add(LSTM(32, input_shape=(784, 1))) # LSTM
model.add(Dense(10, activation='softmax')) # Classification Outputs

model.compile(loss='categorical_crossentropy',
              optimizer='rmsprop',
              metrics=['accuracy'])
model.summary()
early_stop = EarlyStopping(monitor='loss', patience=3, verbose=1)
history = model.fit(
    X_train, y_train_cat,
    epochs=100, batch_size=32, verbose=1,
    validation_split=0.25, shuffle=True,
    callbacks=[early_stop])<jupyter_output><empty_output><jupyter_text>Evaluate<jupyter_code>result = model.evaluate(X_test, y_test_cat)

display(result)
print('Accuracy: ' + str(result[1]*100.0) + ' %')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12855051244509516, 'fraction_numerical': 0.027964860907759882, 'mean_word_length': 6.632402234636872, 'pattern_counts': {'":': 0, '<': 49, '<?xml version=': 0, '>': 50, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '528402', 'license': 'Unlicense', 'n_tokens_mistral': 2670, 'n_tokens_neox': 2505, 'n_words': 576, 'path': 'CourseUdemy_Keras/5_RNN/4_Exercises.ipynb', 'repo_name': 'dangkhoa1992/Advanced-Machine-Learning-Specialization', 'size': '6830'}