@inproceedings{
abrevaya2023effective,
title={Effective Latent Differential Equation Models via Attention and Multiple Shooting},
author={Germ{\'a}n Abrevaya and Mahta Ramezanian-Panahi and Jean-Christophe Gagnon-Audet and Pablo Polosecki and Irina Rish and Silvina Ponce Dawson and Guillermo Cecchi and Guillaume Dumas},
booktitle={The Symbiosis of Deep Learning and Differential Equations III},
year={2023},
url={https://openreview.net/forum?id=RFFTb7DIqi}
}
The GOKU-net is a continuous-time generative model that allows leveraging prior knowledge in the form of differential equations. We present GOKU-UI, an evolution of the GOKU-nets, which integrates attention mechanisms and a novel multiple shooting training strategy in the latent space. On simulated data, GOKU-UI significantly improves performance in reconstruction and forecasting, outperforming baselines even with 16 times less training data. Applied to empirical human brain data, using stochastic Stuart-Landau oscillators, it is able to effectively capture complex brain dynamics, surpassing baselines in reconstruction and better predicting future brain activity up to 15 seconds ahead. Ultimately, our research provides further evidence on the fruitful symbiosis given by the combination of established scientific insights and modern machine learning.
Effective Latent Differential Equation Models via Attention and Multiple Shooting
Germán Abrevaya Mahta Ramezanian-Panahi Jean-Christophe Gagnon-Audet Pablo Polosecki Irina Rish Silvina Ponce Dawson Guillermo Cecchi Guillaume Dumas