We implement the proposed algorithm in PyTorch and use Vimeo90K [1]} training set to train FGDCN. We use Adam optimizer to optimize the network. The training process contains two stages. In the first stage, we train the flow step using \(\mathcal {L}_{fstep}\)  for \(2\times 10^{5}\)  iterations with batch size 24. The learning rate is set to \(1\times 10^{-4}\) . In the second stage, we train the whole model using \(\mathcal {L}_{total}\)  in an end-to-end manner for \(4\times 10^{5}\)  iterations with batch size 32. The learning rate is initially set to \(1\times 10^{-4}\) , and decays to \(1\times 10^{-5}\)  using a cosine attenuation schedule.
We also attempt to train the entire network directly in an end-to-end manner, but it will lead to slight performance degradation (discuss in ablation study).
The weight coefficients \(\lambda _{pf}\) , \(\lambda _{dis}\)  and \(\lambda _{sen}\)  are 0.1, 0.01 and 1, respectively. We randomly crop \(128\times 128\)  patches from the training samples and augment them by random flip and time reversal.
