Deep Discriminative Direct Decoders for High-dimensional Time-series Analysis
Dynamical latent variable modeling has been significantly invested over the
last couple of decades with established solutions encompassing generative
processes like the state-space model (SSM) and discriminative processes like a
recurrent or a deep neural network (DNN). These solutions are powerful tools
with promising results; however, surprisingly they were never put together in a
unified model to analyze complex multivariate time-series data. A very recent
modeling approach, called the direct discriminative decoder (DDD) model,
proposes a principal solution to combine SMM and DNN models, with promising
results in decoding underlying latent processes, e.g. rat movement trajectory,
through high-dimensional neural recordings. The DDD consists of a) a state
transition process, as per the classical dynamical models, and b) a
discriminative process, like DNN, in which the conditional distribution of
states is defined as a function of the current observations and their recent
history. Despite promising results of the DDD model, no training solutions, in
the context of DNN, have been utilized for this model. Here, we propose how DNN
parameters along with an optimal history term can be simultaneously estimated
as a part of the DDD model. We use the D4 abbreviation for a DDD with a DNN as
its discriminative process. We showed the D4 decoding performance in both
simulation and (relatively) high-dimensional neural data. In both datasets, D4
performance surpasses the state-of-art decoding solutions, including those of
SSM and DNNs. The key success of DDD and potentially D4 is efficient
utilization of the recent history of observation along with the state-process
that carries long-term information, which is not addressed in either SSM or DNN
solutions. We argue that D4 can be a powerful tool for the analysis of
high-dimensional time-series data.