Recurrent Neural Networks for Multimodal Time Series Big Data Analytics
This chapter considers the challenges when using Recurrent Neural Networks (RNNs) for Big multimodal time series, forecasting where both the spatial and temporal information has to be used for accurate forecasting. Although RNN and its variations such as Long Short Term Memory (LSTM) Gated Recurrent Unit (GRU) and Simple Recurrent Unit (SRU) progressively improve the quality of training outcomes by implementing the network structures and optimisation techniques, one major limitation in such models is that most of them vectorise the input data and thus destroy a continuous spatial representation. We propose an approach termed the Tensorial Recurrent Neural Network (TRNNs) which addresses the problem on how to analyse the multimodal data as well as considers the relationship along with the time series, and shows that the TRNN outperforms other RNN models for image captioning applications.