
We introduce a differentiable fusion model enabling to solve various temporal settings. We consider three fundamental cases, i.e., interpolation, prediction and motion deblur, which can serve as building blocks for other complex settings. The problem can be solved by automatic differentiation. We refer to this process as Differentiable Model-based Reconstruction (DMR).

We introduce a novel event binning strategy and compare it against conventional stacking-based binning strategy [1]}, [2]}, [3]}, [4]}. Our binning preserves the temporal information of events necessary for high frame-rate video reconstruction. Additionally, we perform statistical evaluation for our binning strategy on the existing dataset [5]}.

We introduce a deep learning strategy for further improving the DMR results. We model the DMR artifacts as additive “noise” and perform “denoising” via deep residual learning. During training, we augment the samples by randomizing all the parameters of the DMR. We show preliminary results that the trained residual denoiser (RD) has properties including constrast enhancement and motion awareness, which is beyond a Gaussian denoiser.

<FIGURE>