<jupyter_start><jupyter_text>Pre-processing Data Standardization**Normally distrbuted data** - Gaussian with zero mean and unit varianceMany elements used in the objective function of a learning algorithm (such as the RBF kernel of SVM) assume that all features are centered around zero and have variance in the same order.You can use the **scale** function to achieve zero mean and variance data:<jupyter_code>from sklearn import preprocessing
import numpy as np
scaled = preprocessing.scale(train.view)

plt.plot(train.time, scaled)
plt.show()<jupyter_output><empty_output><jupyter_text>You can use a **Transformer** to record compute the mean and sd and later apply the transformation on the testing set.  Note that *fit()* expected a 2D numpy array not a 1D array.  You must call reshape on the 1D array first before passing it to fit.<jupyter_code>X_train = train.view.values.reshape(-1, 1)
scaler = preprocessing.StandardScaler().fit(X_train)
print(scaler.mean_)
print(scaler.scale_)
scaler.transform(X_train)[0:10]<jupyter_output>[10.0182429]
[1.11076688]<jupyter_text>Simliarly, there are some other useful encoders that records the transformation and can be re-applied to another dataset later<jupyter_code>import datetime
from sklearn.preprocessing import LabelEncoder
from sklearn.preprocessing import OneHotEncoder

#weekdays = [d.strftime('%a')for d in train.time]
#weekdays_labels = LabelEncoder().fit_transform(weekdays).reshape(-1, 1)
#print(weekdays_labels[0:5])

weekdays_encoded = OneHotEncoder(sparse=False).fit_transform(
    np.array(weekdays).reshape(-1, 1))
print(weekdays_encoded[0:5])
# np.expand_dims(weekdays_encoded,0)
# Create an array to discover lagged features

# Some functions for auto correlations
def lag_arr(arr, lag, fill):
    filler = np.full((arr.shape[0],lag,1),-1)
    comb = np.concatenate((filler,arr),axis=1)
    result = comb[:,:arr.shape[1]]
    return result

def single_autocorr(series, lag):
    s1 = series[lag:]
    s2 = series[:-lag]
    ms1 = np.mean(s1)
    ms2 = np.mean(s2)
    ds1 = s1 - ms1
    ds2 = s2 - ms2
    divider = np.sqrt(np.sum(ds1 * ds1)) * np.sqrt(np.sum(ds2 * ds2))
    return np.sum(ds1 * ds2) / divider if divider != 0 else 0

def batc_autocorr(data,lag,series_length):
    corrs = []
    for i in range(data.shape[0]):
        c = single_autocorr(data, lag)
        corrs.append(c)
    corr = np.array(corrs)
    corr = np.expand_dims(corr,-1)
    corr = np.expand_dims(corr,-1)
    corr = np.repeat(corr,series_length,axis=1)
    return corr<jupyter_output><empty_output><jupyter_text>Conv1DSimilar to 2D Convolution Neural Networks, there is also aversion of convolutional filters that can slide over a sequence one-dimensionally.<jupyter_code>n_features = 29
max_len = 100


# Compile the model
model = Sequential()
model.add(Conv1D(16,5, input_shape=(100,29)))
model.add(Activation('relu'))
model.add(MaxPool1D(5))

model.add(Conv1D(16,5))
model.add(Activation('relu'))
model.add(MaxPool1D(5))
model.add(Flatten()) # Transforms a tensor into a flat tensor 
model.add(Dense(1))
model.compile(optimizer='adam',loss='mean_absolute_percentage_error')


# Train the model
from sklearn.model_selection import train_test_split

batch_size = 128
train_df, val_df = train_test_split(train, test_size=0.1)
train_gen = generate_batches(train_df,batch_size=batch_size)
val_gen = generate_batches(val_df, batch_size=batch_size)

n_train_samples = train_df.shape[0]
n_val_samples = val_df.shape[0]<jupyter_output><empty_output><jupyter_text>RNN Recurrent Neural Network - This network contains recurrent layers that can remember their last activation and use it as their own input.  A recurrent layer takes a sequence as an input.  For each element, it then computes a matrix multiplication (like a Dense layer), and runs the results through an activation such as relu.<jupyter_code>from keras.layers import SimpleRNN

# Compile the model
model = Sequential()
model.add(SimpleRNN(16,return_sequences=True,input_shape=(max_len,n_features)))
model.add(SimpleRNN(32, return_sequences = True))
model.add(SimpleRNN(64))
model.add(Dense(1))
model.compile(optimizer='adam',loss='mean_absolute_percentage_error')

# Trains the model on data generated batch-by-batch by a Python generator (or an instance of Sequence).

model.fit_generator(train_gen,epochs=20,steps_per_epoch=n_train_samples // batch_size, validation_data= val_gen, validation_steps=n_val_samples // batch_size)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10543989254533244, 'fraction_numerical': 0.02350570852921424, 'mean_word_length': 5.7799696509863425, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 20, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '111545', 'license': 'Apache-2.0', 'n_tokens_mistral': 1550, 'n_tokens_neox': 1482, 'n_words': 396, 'path': 'klaas_machine_learning/time_series_forecasting_nn.ipynb', 'repo_name': 'hungapl/python', 'size': '4467'}