Data preparation.   We use publicly available high-speed (240 FPS) video dataset, the Need for Speed dataset [1]}. The reason we choose this dataset is because it has rich motion categories and content (100 videos with 380K frames) which involves both camera and scene/object motion. As introduced in Section REF , our RD is trained on the output of DMR process. As a proof of concept, we simulate solving a single-frame prediction problem, i.e. given two consecutive video frames, we first simulate the latent event frame. Next, a DMR is performed to predict the end frame.
