<jupyter_start><jupyter_text>Building a Multiple Input Multi-step output- Use Func to split timeseries in to 6 steps in, 6 steps out<jupyter_code>train = pd.read_csv('../data/TrainingSet_Cleaned.csv',index_col=0)
submission = pd.read_csv('../data/SubmissionRows.csv',index_col=0)
train.head()<jupyter_output><empty_output><jupyter_text>Prepare data<jupyter_code># split a multivariate sequence into samples
def split_sequences(sequences, n_steps_in, n_steps_out):
	X, y = list(), list()
	for i in range(len(sequences)):
		# find the end of this pattern
		end_ix = i + n_steps_in
		out_end_ix = end_ix + n_steps_out
		# check if we are beyond the dataset
		if out_end_ix > len(sequences):
			break
		# gather input and output parts of the pattern: target is the next n_steps_out rows at the end of n_steps_in
		seq_x, seq_y = sequences[i:end_ix, :], sequences[end_ix:out_end_ix, :]
		X.append(seq_x)
		y.append(seq_y)
	return np.array(X), np.array(y)
# Scaler:
scaler = MinMaxScaler()

# Train, Test sets
data = train.loc[:,'1980':'2007'].T

data = scaler.fit_transform(data)

# Back to DF: Consider timeframe 1980 >
data = pd.DataFrame(data,index=train.loc[:,'1980':'2007'].columns,columns=train.loc[:,'1980':'2007'].index)

### Fill rest of nan values with -1:
data = data.fillna(-1)

# Back to numpy:
data = data.values

# Timesteps as input/Output:
n_steps_in =5
n_steps_out = 5

X, y = split_sequences(data,n_steps_in,n_steps_out)

n_features = X.shape[2]
# Shape of X, y:
print('Shape of X : ',X.shape)
print('Shape of y : ',y.shape) # 737 rows to predict<jupyter_output>Shape of X :  (19, 5, 4068)
Shape of y :  (19, 5, 4068)<jupyter_text>LSTM Network:Multivariate network, multi-output, predict the 2008 for the submission years- Predicts 5 timesteps ahead (years)- Encoder - Decoder LSTM model- Wrapping output into a TimeDistributed layer- 2 Layer, 200 hidden units- Train 1000 Epochs, validation split 20% (training data)- Early stopping round (patience = 50)- Save best model with ModelCheckpointGoal: To predict 2008 (Target), 2009, 2010, 2011, 2012 (Target)  Build model:- Encoder ,Decoder<jupyter_code># Define model: 128 units
model = models.Sequential()
model.add(layers.Masking(mask_value=-1,input_shape=(n_steps_in,n_features)))
model.add(layers.LSTM(128, activation='relu'))
model.add(layers.RepeatVector(n_steps_out))
model.add(layers.LSTM(128, activation='relu', return_sequences=True))
model.add(layers.TimeDistributed(layers.Dense(y.shape[2])))
# compile:
model.compile(optimizer='adam',loss='mse',metrics=['mean_absolute_error'])

# Early stopping round, Model Checkpoint
es = callbacks.EarlyStopping(monitor='val_mean_absolute_error',mode='min',patience=200,verbose=1)
mc = callbacks.ModelCheckpoint('ts_un.h5',save_best_only=True,monitor='val_mean_absolute_error',mode='min',verbose=1)

# Fit:
# history = model.fit(X,y,validation_split=0.1,epochs=2000,shuffle=False,callbacks=[es,mc])
history = model.fit(X,y,validation_split=0.1,epochs=10,shuffle=False,callbacks=[es,mc])
#plot:
plt.figure(figsize=(10,8))
plt.plot(history.history['val_loss'],label='val_loss')
plt.plot(history.history['loss'],label='loss')
plt.legend()<jupyter_output><empty_output><jupyter_text>Predict<jupyter_code># X_test:
X_input = data[-5:,:].reshape(1,5,data.shape[1])
# Predict: Load Model
saved_model = models.load_model('ts_un.h5')
y_pred_2008_2012 = saved_model.predict(X_input)
y_pred_2008_2012 = y_pred_2008_2012.reshape(5,y_pred_2008_2012.shape[2])
# Build back DF:
index_years = list(train.loc[:,'1980':'2007'].columns) + [str(x) for x in range(2008,2013)]
columns = list(train.loc[:,'1980':'2007'].index)
df_1972_2012 = pd.DataFrame(np.concatenate([data,y_pred_2008_2012]),index=index_years,columns=columns)

# Put into right format
df_1972_2012_unscaled = scaler.inverse_transform(df_1972_2012.values)

df_1972_2012_unscaled = pd.DataFrame(df_1972_2012_unscaled,index=index_years,columns=columns)

df_1972_2012_unscaled = df_1972_2012_unscaled.T


# Incase values became negative:
df_1972_2012_unscaled = df_1972_2012_unscaled[['2008','2012']].applymap(lambda x: abs(x))
# Submission columns:
df_pred = df_1972_2012_unscaled.loc[submission.index,:]

df_pred.columns = submission.columns
df_pred.to_csv('../bin/Final_Pred.csv')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14031389084094636, 'fraction_numerical': 0.06558913094401499, 'mean_word_length': 7.180076628352491, 'pattern_counts': {'":': 0, '<': 16, '<?xml version=': 0, '>': 18, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '350581', 'license': 'MIT', 'n_tokens_mistral': 1804, 'n_tokens_neox': 1552, 'n_words': 311, 'path': 'LSTM_EXPERIMENTS/2_LSTM_Keras.ipynb', 'repo_name': 'machine-learning-study-group/time-series-millenium-goals', 'size': '4269'}