<jupyter_start><jupyter_text>Series de tiempo cierre acciones - LSTM y PCA Con la ayuda de Principal Component Analysis (PCA) se analizarán las características del dataset, con el fin de revisar su variabilidad y la posibiidad de reducir sus características. Luego, se crearán diferentes tipos de modelos de arquitectura LSTM, donde varia el número de neuronas de la capa LSTM y el lookback para realizar el mismo ejercicio de los anteriores notebooks: Entrenamiento de los modelos, selección del mejor modelo y predicciones con éste último.  Contenido  1. Importación de librerías y cargue del dataset 2. PCA 3. Definión funciones de creación de dataset y gráficos de los modelos 4. LSTM  1. Importación de librerías y cargue del dataset A continuación, se cargan las librerías necesarias para completar esta tarea:<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import itertools
import tensorflow as tf
from sklearn.preprocessing import MinMaxScaler
from sklearn.metrics import mean_squared_error
from sklearn import decomposition
from keras.layers import Dense, LSTM, RepeatVector, TimeDistributed
from keras.models import Sequential
%matplotlib inline
Se lee el archivo de excel que contiene la información del dataset y se convierte la columna fecha en el id de cada final.
!mkdir ./data
!mv Dataset.xls ./data
dataset = pd.read_excel('./data/Dataset.xls')
dataset.set_index('fecha',inplace=True)
dataset = dataset.drop(dataset.columns[[0]], axis=1)
dataset.head()<jupyter_output><empty_output><jupyter_text>Los componentes de la transformada de Fourier se convierten al formato de número complejo, debido a que al leerlos del archivo quedan como texto.<jupyter_code>dataset['Fourier3'] = np.asarray(dataset['Fourier3'].tolist()).astype('complex').real
dataset['Fourier6'] = np.asarray(dataset['Fourier6'].tolist()).astype('complex').real
dataset['Fourier9'] = np.asarray(dataset['Fourier9'].tolist()).astype('complex').real
dataset['Fourier100'] = np.asarray(dataset['Fourier100'].tolist()).astype('complex').real<jupyter_output><empty_output><jupyter_text>2. PCA Se normaliza el dataset.<jupyter_code>scaler = MinMaxScaler(feature_range=(0, 1))
scaler.fit(dataset)
data_scaled = scaler.transform(dataset)
data_scaled.shape<jupyter_output>/usr/local/lib/python2.7/dist-packages/sklearn/preprocessing/data.py:334: DataConversionWarning: Data with input dtype int64, float64 were all converted to float64 by MinMaxScaler.
  return self.partial_fit(X, y)<jupyter_text>Se trasnforman los datos con PCA, donde se observa que la dimensión del dataset transformado es igual a la original.<jupyter_code>n_components = min(len(data_scaled), data_scaled.shape[1])
pca = decomposition.PCA(n_components=n_components)
pca.fit(data_scaled)
data_pca = pca.transform(data_scaled)
data_pca.shape<jupyter_output><empty_output><jupyter_text>A continuación se revisa la varianza de los datos con PCA, encontrando  que con 6 componentes tenemos algo mas del 94% de varianza explicada.<jupyter_code>expl_var = pca.explained_variance_ratio_
print(expl_var)
print('porcentaje varianza:',sum(expl_var[0:6]))<jupyter_output>[7.53139121e-01 7.82956869e-02 5.38278493e-02 2.52394736e-02
 2.07532328e-02 1.69722329e-02 1.29602872e-02 1.01908574e-02
 8.31646415e-03 6.90189222e-03 5.04726965e-03 2.47615839e-03
 1.85579774e-03 1.31728648e-03 1.01879394e-03 4.94830156e-04
 4.21980538e-04 2.76929602e-04 2.13662178e-04 1.72155984e-04
 4.76635320e-05 2.37482027e-05 1.68691148e-05 1.03337317e-05
 6.98214224e-06 2.12403871e-06 3.16936600e-07 1.48530502e-31
 1.17306637e-32 5.00460557e-33 4.28176157e-33]
('porcentaje varianza:', 0.9482275967173559)<jupyter_text>Se grafica el acumulado de varianza explicada.<jupyter_code>plt.plot(np.cumsum(pca.explained_variance_ratio_))
plt.xlabel('number of components')
plt.ylabel('cumulative explained variance')
plt.show()<jupyter_output><empty_output><jupyter_text>3. Definión funciones de creación de dataset y gráficos de los modelos Función para construir el dataset con múltiples características para train y test de acuerdo al lookback, time_ahead y caractetística a predecir.<jupyter_code>def create_dataset_nfeatures(dataset, n_steps_out=1, n_steps_in = 1, overlap = 1, index_feature_pred = None):
    dataX, dataY = [], []
    tem = n_steps_in + n_steps_out - overlap
    for i in range(int((len(dataset) - tem)/overlap)):
        startx = i*overlap
        endx = startx + n_steps_in
        starty = endx
        endy = endx + n_steps_out
        a = dataset[startx:endx, :]
        dataX.append(a)
        dataY.append(dataset[starty:endy,index_feature_pred])
    return np.array(dataX), np.array(dataY)<jupyter_output><empty_output><jupyter_text>Función que grafica la pérdida mse de diferentes modelos.<jupyter_code>def plot_metrics_model(mse, combinations, title):
    plt.figure(figsize=(20,8))
    plt.plot(mse, lw=4, alpha=.5, label="MSE models")
    plt.grid();
    plt.xlabel("Number of steps in")
    plt.ylabel("MSE")
    plt.title(title)    
    plt.xticks(np.arange(len(combinations)), combinations, rotation=90)
    plt.legend();
    plt.show()<jupyter_output><empty_output><jupyter_text>Función que grafica las predicciones de un modelo con la serie original.<jupyter_code>def plot_predictions_model(x_lab1, Original_serie, x_lab2, trainPredict, x_lab3, testPredict, title):
    plt.plot(x_lab1,Original_serie, alpha=1, label='Original series');
    plt.plot(x_lab2,trainPredict,alpha=0.6, label='train prediction');
    plt.plot(x_lab3,testPredict, alpha=0.6, label='test prediction');
    plt.title(title);
    plt.legend();
    plt.show()<jupyter_output><empty_output><jupyter_text>Se crean los datos de train y test proveniendo del dataset transformado por PCA, el cual ya está normalizado. Sólo se toman las primeras 6 características, de acuerdo a la varianza representada en el análisis de PCA.<jupyter_code>train_size = int(len(data_pca) * 0.67)
test_size = len(data_pca) - train_size
trainN, testN = data_pca[0:train_size,0:6], data_pca[train_size:len(data_pca),0:6]
trainN.shape, testN.shape<jupyter_output><empty_output><jupyter_text>En la siguiente celda, se crean varios datasets de train y test donde para diferentes lookback. Se define time_ahead igual a 1 para predecir un solo movimiento y se selecciona la caractetística a predecir.<jupyter_code>time_ahead=1

X_train=[]
y_train=[]
X_test=[]
y_test=[]
for i in range (10):
    X_trainTemp, y_trainTemp = create_dataset_nfeatures(trainN, time_ahead,i+1,index_feature_pred=2)
    X_testTemp, y_testTemp = create_dataset_nfeatures(testN, time_ahead,i+1,index_feature_pred=2)
    X_train.append(X_trainTemp),y_train.append(y_trainTemp),X_test.append(X_testTemp), y_test.append(y_testTemp)
len(X_train),X_train[0].shape,len(y_train),y_train[0].shape<jupyter_output><empty_output><jupyter_text>Se definen las listas para crear diferentes tipos de modelos, donde se combinarán diferentes dimesiones de la capa LSTM y lookbacks.<jupyter_code>ltsm_dim_set = [64,128,196,256]
look_back_set = [1,2,3,4,5,6,7,8,9,10]<jupyter_output><empty_output><jupyter_text>4. LSTM Función para crear, entrenar y calcular pérdida mse de modelos de arquitectura LTSM.<jupyter_code>def LSTM_model(ltsm_dim, X_train, y_train, X_test, y_test, look_back):
    X_train=X_train[look_back-1]
    y_train=y_train[look_back-1]
    X_test=X_test[look_back-1]
    y_test=y_test[look_back-1]
    
    model = Sequential()    
    model.add(LSTM(ltsm_dim, input_shape=(look_back, X_train.shape[2])))
    model.add(Dense(1))
    model.compile(optimizer='adam',loss='mse')
    
    print "Entrenado modelo LSTM con look_back de", look_back, "y neuronas", ltsm_dim
    model.fit(X_train, y_train.flatten(),epochs=50, verbose=1)
    pred = model.predict(X_test)
    mse = mean_squared_error(y_test,pred)
    return model, mse<jupyter_output><empty_output><jupyter_text>Se crean los modelos LSTM con diferentes número de neuronas de la capa LSTM y lookback.<jupyter_code>LSTM_models=[]
LSTM_mse=[]
LSTM_combinations = []
for ltsm_dim, look_back in itertools.product(ltsm_dim_set, look_back_set):
    model, mse = LSTM_model(ltsm_dim, X_train, y_train, X_test, y_test,look_back=look_back)
    LSTM_models.append(model), LSTM_mse.append(mse)
    LSTM_combinations.append('lstm '+str(ltsm_dim)+' - look_back '+str(look_back))<jupyter_output>WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
Entrenado modelo LSTM con look_back de 1 y neuronas 64
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
Epoch 1/50
1833/1833 [==============================] - 3s 2ms/step - loss: 0.0462
Epoch 2/50
1833/1833 [==============================] - 0s 157us/step - loss: 0.0192
Epoch 3/50
1833/1833 [==============================] - 0s 157us/step - loss: 0.0063
Epoch 4/50
1833/1833 [==============================] - 0s 159us/step - loss: 0.0032
Epoch 5/50
1833/1833 [==============================] - 0s 1[...]<jupyter_text>Se grafica la pérdida mse de los modelos LSTM.<jupyter_code>plot_metrics_model(LSTM_mse,LSTM_combinations, "Performance using LSTM - data from PCA")<jupyter_output><empty_output><jupyter_text>Se muestra la pérdida mse de los modelos LSTM.<jupyter_code>columns = ['model', 'mse']
df_models_lstm = pd.DataFrame(columns=columns)
df_models_lstm['model'] = LSTM_combinations
df_models_lstm['mse'] = LSTM_mse
df_models_lstm<jupyter_output><empty_output><jupyter_text>Elegimos el modelo LSTM de menor pérdida mse.<jupyter_code>index_model_min_mse = df_models_lstm[df_models_lstm.mse == df_models_lstm.mse.min()].index[0]
look_back = df_models_lstm.loc[index_model_min_mse,'model']
look_back = int(look_back[look_back.rfind(" ")+1:])
print index_model_min_mse, look_back
df_models_lstm[df_models_lstm.mse == df_models_lstm.mse.min()]<jupyter_output>0 1<jupyter_text>Realizamos las predicciones con el modelo LSTM seleccionado.<jupyter_code>model_lstm = LSTM_models[index_model_min_mse]
model_lstm.summary()
trainPredict = model_lstm.predict(X_train[look_back-1])
testPredict = model_lstm.predict(X_test[look_back-1])
trainPredict.shape, testPredict.shape
Original_serie = scaler.fit_transform(data_pca[:,2].reshape(-1, 1))
Original_serie.shape
# data_pca_inversa = pca.inverse_transform(data_pca)
# Original_serie = scaler.fit_transform(data_pca_inversa[:,2].reshape(-1, 1))
# data_pca_inversa.shape, Original_serie.shape
x_lab1=[]
for i in range(Original_serie.shape[0]):
    x_lab1.append(i)
x_lab2=[]
for i in range(trainPredict.shape[0]):
    x_lab2.append(i)
x_lab3=[]
for i in range(trainPredict.shape[0]+9,Original_serie.shape[0]+7):
    x_lab3.append(i)
    
len(x_lab1), Original_serie.shape[0],len(x_lab2), trainPredict.shape[0],len(x_lab3), testPredict.shape[0]<jupyter_output><empty_output><jupyter_text>Se grafican las predicciones del modelo LSTM con los datos transformados de PCA.<jupyter_code>plot_predictions_model(x_lab1, 
                       Original_serie, 
                       x_lab2, 
                       trainPredict, 
                       x_lab3, 
                       testPredict, 
                       "LSTM predictions data PCA for look_back = "+ str(look_back))<jupyter_output><empty_output><jupyter_text>Se exporta el modelo LSTM entrenado con el dataset transformado por PCA.<jupyter_code>model_dir = './models/'
tf.gfile.MakeDirs(model_dir)
model_name = df_models_lstm.loc[index_model_min_mse,'model']
model_lstm.save(model_dir+model_name+"_PCA.model")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.117191446371443, 'fraction_numerical': 0.051355447044957064, 'mean_word_length': 5.854587420657819, 'pattern_counts': {'":': 0, '<': 77, '<?xml version=': 0, '>': 77, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '20', 'dir': 'jupyter-structured-clean-dedup', 'id': '299186', 'license': 'MIT', 'n_tokens_mistral': 4765, 'n_tokens_neox': 4404, 'n_words': 926, 'path': 'U4.04 - Series de tiempo cierre acciones - LSTM y PCA.ipynb', 'repo_name': 'Jesda03/RNN-Closing-Stock-Price-Predictions-ESP', 'size': '11878'}