<jupyter_start><jupyter_text>Forcasting using spatio-temporal data with combined Graph Convolution + LSTM modelThe dynamics of many real-world phenomena are spatio-temporal in nature. Traffic forecasting is a quintessential example of spatio-temporal problems for which we present here a deep learning framework that models speed prediction using spatio-temporal data. The task is challenging due to two main inter-linked factors: (1) the complex spatial dependency on road networks, and (2) non-linear temporal dynamics with changing road conditions.To address these challenge, here we  explore a neural network architecture that learns from both the spatial road network data and time-series of historical speed changes to forecast speeds on road segments at a future time. We use a 2-layer graph convolution network to leverage the graph structure of the road network. The output of the graph convolution network is fed into an LSTM based sequence to sequence model, that is jointly trained on the output of gcn and the historical speeds of a segment. The architecture of the gcn-lstm model is inpired by the paper: [T-GCN: A Temporal Graph Convolutional Network for Traffic Prediction](https://ieeexplore.ieee.org/document/8809901).The authors have made available the implementation of their model in their github [repo](https://github.com/lehaifeng/T-GCN).There has been a few differences in the architecture proposed in the paper and the implementation with regards to the graph convolution component, these issues have been documented [here](https://github.com/lehaifeng/T-GCN/issues/18) and [here](https://github.com/lehaifeng/T-GCN/issues/14). Here we attempt at emulating the model as explained in the paperr. Concretetly, 1. We use the 2-layer graph convolutional network (gcn) proposed by [Kipf & Welling (ICLR 2017)](http://arxiv.org/abs/1609.02907)2. We stack the gcn with 1 layer of LSTM. The [TGCN](https://ieeexplore.ieee.org/document/8809901) uses GRU instead of LSTM. In practice there are not any remarkable differences between the two types of layers. We use LSTM as they are more frequently used.3. We further add a Dropout and Dense layer as they experimentally showed improvement in performance and managing over-fitting. References: * [T-GCN: A Temporal Graph Convolutional Network for Traffic Prediction](https://ieeexplore.ieee.org/document/8809901)* [https://github.com/lehaifeng/T-GCN](https://github.com/lehaifeng/T-GCN)* [Semi-Supervised Classification with Graph Convolutional Networks](http://arxiv.org/abs/1609.02907) Run the master version of this notebook:<jupyter_code># install StellarGraph if running on Google Colab
import sys
if 'google.colab' in sys.modules:
  %pip install -q stellargraph[demos]
# verify that we're using the correct version of StellarGraph for this notebook
import stellargraph as sg

try:
    sg.utils.validate_notebook_version("1.0.0b")
except AttributeError:
    raise ValueError(
        f"This notebook requires StellarGraph version 1.0.0b, but a different version {sg.__version__} is installed.  Please see <https://github.com/stellargraph/stellargraph/issues/1172>."
    ) from None
import os
import sys
import urllib.request

import numpy as np
import matplotlib.pyplot as plt
import matplotlib.lines as mlines

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import Sequential, Model
from tensorflow.keras.layers import LSTM, Dense, Dropout, Input<jupyter_output><empty_output><jupyter_text>DataWe apply the gcn-lstm model to the ***Los-loop*** data. This traffic datasetcontains traffic information collected from loop detectors in the highway of Los Angeles County (Jagadishet al., 2014).  There are several processed versions of this dataset used by the research community working in Traffic forecasting space. This demo is based on the pre-processed version of the dataset used by the TGCN paper. It can be directly accessed from there [github repo](https://github.com/lehaifeng/T-GCN/tree/master/data). This dataset  contains traffic speeds from Mar.1 to Mar.7, 2012 of 207 sensors, recorded every 5 minutes. In order to use the model, we need:* A N by N adjacency matrix, which describes the spatial relationship between roads,* A N by D feature matrix, which describes the speed change over time on the roads.We have provided the csv files of the weighted adjacency matrix and the time-series of speeds in the [/data](/data) directory with the demo for convenience. However, this may change in near future. The data can still be obtained by the TGCN repo nonetheless. A couple of other references for the same data albeit different time length are as follows: * [DIFFUSION CONVOLUTIONAL RECURRENT NEURAL NETWORK: DATA-DRIVEN TRAFFIC FORECASTING](https://github.com/liyaguang/DCRNN/tree/master/data): This dataset consists of 207 sensors and collect 4 months of data ranging from Mar 1st 2012 to Jun 30th 2012 for the experiment. It has some missing values.* [ST-MetaNet: Urban Traffic Prediction from Spatio-Temporal Data Using Deep Meta Learning](https://github.com/panzheyi/ST-MetaNet/tree/master/traffic-prediction). This work uses the DCRNN pre-proccessed data.  Loading and pre-processing the data<jupyter_code>import stellargraph as sg
dataset = sg.datasets.METR_LA()<jupyter_output><empty_output><jupyter_text>This demo is based on the pre-processed version of the dataset used by the TGCN paper. The data needs to be downloaded from: [https://github.com/lehaifeng/T-GCN/tree/master/data] for this demo and placed in a 'data' folder within the working directory.<jupyter_code>data, adj = dataset.load()
num_nodes = data.shape[1]
time_len = data.shape[0]
print("No. of sensors:", num_nodes, "\nNo of timesteps:", time_len)<jupyter_output>No. of sensors: 207 
No of timesteps: 2016<jupyter_text>Train/test split<jupyter_code>train_rate = 0.8
train_data, test_data = dataset.train_test_split(data, train_rate)
print("Train data: ", train_data.shape)
print("Test data: ", test_data.shape)<jupyter_output>Train data:  (1612, 207)
Test data:  (404, 207)<jupyter_text>Scaling<jupyter_code>train_scaled, test_scaled = dataset.scale_data(train_data, test_data)<jupyter_output><empty_output><jupyter_text>Sequence data perparation for LSTM* Each training observation are 10 historical speeds.*Each training prediction is the speed 30 minutes later.<jupyter_code>seq_len = 10
pre_len = 12
trainX, trainY, testX, testY = dataset.sequence_data_preparation(
    seq_len, pre_len, train_scaled, test_scaled
)
print(trainX.shape)
print(trainY.shape)
print(testX.shape)
print(testY.shape)<jupyter_output>(1591, 10, 207)
(1591, 207)
(383, 10, 207)
(383, 207)<jupyter_text>StellarGraph Graph Convolution and LSTM model<jupyter_code>from stellargraph.layer import GraphConvolutionLSTM
gcn_lstm = GraphConvolutionLSTM(
    seq_len=seq_len, adj=adj, gc_layers=2, lstm_layer_sizes=[100]
)
x_input, x_output = gcn_lstm.in_out_tensors()
model = Model(inputs=x_input, outputs=x_output)
model.compile(optimizer="adam", loss="mae", metrics=["mse"])
history = model.fit(
    trainX,
    trainY,
    epochs=100,
    batch_size=60,
    shuffle=True,
    verbose=0,
    validation_data=[testX, testY],
)
model.summary()
print(
    "Train loss: ",
    history.history["loss"][-1],
    "\nTest loss:",
    history.history["val_loss"][-1],
)
plt.plot(history.history["loss"], label="Training loss")
plt.plot(history.history["val_loss"], label="Test loss")
plt.legend()
plt.xlabel("epoch")
plt.ylabel("loss")
plt.show()
ythat = model.predict(trainX)
yhat = model.predict(testX)<jupyter_output><empty_output><jupyter_text>Rescale values<jupyter_code>## Rescale values
max_speed = train_data.max()
min_speed = train_data.min()

## actual train and test values
train_rescref = np.array(trainY * max_speed)
test_rescref = np.array(testY * max_speed)
## Rescale model predicted values
train_rescpred = np.array((ythat) * max_speed)
test_rescpred = np.array((yhat) * max_speed)<jupyter_output><empty_output><jupyter_text>Measuring the performance of the modelTo understand how well the model is performing, we compare it against a naive benchmark.1. Naive prediction: using the most recently ***observed*** value as the predicted value. Note, that albeit being ***naive*** this is a very strong baseline to beat. Especially, when speeds are recorded at a 5 minutes granularity,  one does not expect many drastic changes within such a short period of time. Hence, for short-term predictions naive is a reasonable good guess.  Naive prediction benchmark (using latest observed value)<jupyter_code>## Naive prediction benchmark (using previous observed value)

testnpred = np.array(testX).transpose(1, 0, 2)[
    -1
]  # picking the last speed of the 10 sequence for each segment in each sample
testnpredc = (testnpred) * max_speed
## Performance measures

seg_mael = []
seg_masel = []
seg_nmael = []

for j in range(testX.shape[-1]):

    seg_mael.append(
        np.mean(np.abs(test_rescref.T[j] - test_rescpred.T[j]))
    )  # Mean Absolute Error for NN
    seg_nmael.append(
        np.mean(np.abs(test_rescref.T[j] - testnpredc.T[j]))
    )  # Mean Absolute Error for naive prediction
    if seg_nmael[-1] != 0:
        seg_masel.append(
            seg_mael[-1] / seg_nmael[-1]
        )  # Ratio of the two: Mean Absolute Scaled Error
    else:
        seg_masel.append(np.NaN)

print("Total (ave) MAE for NN: " + str(np.mean(np.array(seg_mael))))
print("Total (ave) MAE for naive prediction: " + str(np.mean(np.array(seg_nmael))))
print(
    "Total (ave) MASE for per-segment NN/naive MAE: "
    + str(np.nanmean(np.array(seg_masel)))
)
print(
    "...note that MASE<1 (for a given segment) means that the NN prediction is better than the naive prediction."
)
# plot violin plot of MAE for naive and NN predictions
fig, ax = plt.subplots()
# xl = minsl

ax.violinplot(
    list(seg_mael), showmeans=True, showmedians=False, showextrema=False, widths=1.0
)

ax.violinplot(
    list(seg_nmael), showmeans=True, showmedians=False, showextrema=False, widths=1.0
)

line1 = mlines.Line2D([], [], label="NN")
line2 = mlines.Line2D([], [], color="C1", label="Instantaneous")

ax.set_xlabel("Scaled distribution amplitude (after Gaussian convolution)")
ax.set_ylabel("Mean Absolute Error")
ax.set_title("Distribution over segments: NN pred (blue) and naive pred (orange)")
plt.legend(handles=(line1, line2), title="Prediction Model", loc=2)
plt.show()
##all test result visualization
fig1 = plt.figure(figsize=(15, 8))
#    ax1 = fig1.add_subplot(1,1,1)
a_pred = test_rescpred[:, 1]
a_true = test_rescref[:, 1]
plt.plot(a_pred, "r-", label="prediction")
plt.plot(a_true, "b-", label="true")
plt.legend(loc="best", fontsize=10)
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09727010155594894, 'fraction_numerical': 0.0192863132395416, 'mean_word_length': 5.154816513761468, 'pattern_counts': {'":': 0, '<': 36, '<?xml version=': 0, '>': 35, 'https://': 13, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '334027', 'license': 'Apache-2.0', 'n_tokens_mistral': 3528, 'n_tokens_neox': 3284, 'n_words': 1155, 'path': 'demos/spatio-temporal/gcn-lstm-LA.ipynb', 'repo_name': 'LarsNeR/stellargraph', 'size': '10733'}