Neural Integro-Differential Equations
Modeling continuous dynamical systems from discretely sampled observations is
a fundamental problem in data science. Often, such dynamics are the result of
non-local processes that present an integral over time. As such, these systems
are modeled with Integro-Differential Equations (IDEs); generalizations of
differential equations that comprise both an integral and a differential
component. For example, brain dynamics are not accurately modeled by
differential equations since their behavior is non-Markovian, i.e. dynamics are
in part dictated by history. Here, we introduce the Neural IDE (NIDE), a
framework that models ordinary and integral components of IDEs using neural
networks. We test NIDE on several toy and brain activity datasets and
demonstrate that NIDE outperforms other models, including Neural ODE. These
tasks include time extrapolation as well as predicting dynamics from unseen
initial conditions, which we test on whole-cortex activity recordings in freely
behaving mice. Further, we show that NIDE can decompose dynamics into its
Markovian and non-Markovian constituents, via the learned integral operator,
which we test on fMRI brain activity recordings of people on ketamine. Finally,
the integrand of the integral operator provides a latent space that gives
insight into the underlying dynamics, which we demonstrate on wide-field brain
imaging recordings. Altogether, NIDE is a novel approach that enables modeling
of complex non-local dynamics with neural networks.