E2V-SDE: From Asynchronous Events to Fast and Continuous Video Reconstruction via Neural Stochastic Differential Equations
Event cameras respond to brightness changes in the scene asynchronously and
independently for every pixel. Due to the properties, these cameras have
distinct features: high dynamic range (HDR), high temporal resolution, and low
power consumption. However, the results of event cameras should be processed
into an alternative representation for computer vision tasks. Also, they are
usually noisy and cause poor performance in areas with few events. In recent
years, numerous researchers have attempted to reconstruct videos from events.
However, they do not provide good quality videos due to a lack of temporal
information from irregular and discontinuous data. To overcome these
difficulties, we introduce an E2V-SDE whose dynamics are governed in a latent
space by Stochastic differential equations (SDE). Therefore, E2V-SDE can
rapidly reconstruct images at arbitrary time steps and make realistic
predictions on unseen data. In addition, we successfully adopted a variety of
image composition techniques for improving image clarity and temporal
consistency. By conducting extensive experiments on simulated and real-scene
datasets, we verify that our model outperforms state-of-the-art approaches
under various video reconstruction settings. In terms of image quality, the
LPIPS score improves by up to 12% and the reconstruction speed is 87% higher
than that of ET-Net.