Training details.
We explore different hyper-parameters settings empirically. In the following experiments, we train our method on 4 Tesla V100 16Gb GPUs and adopt a batch size (per GPU) of 32. For updating the gradients, we opt for Adam optimizer [1]} with learning rate of \(0.0003\) . We interrupt the train at local convergence through an early-stopping strategy. We evaluate our approach with 1 stage (7M of parameters, 91 MB of storage) and 3 stages (21M parameters, 300MB of storage).
