Residual Multiplicative Filter Networks for Multiscale Reconstruction
Coordinate networks like Multiplicative Filter Networks (MFNs) and BACON
offer some control over the frequency spectrum used to represent continuous
signals such as images or 3D volumes. Yet, they are not readily applicable to
problems for which coarse-to-fine estimation is required, including various
inverse problems in which coarse-to-fine optimization plays a key role in
avoiding poor local minima. We introduce a new coordinate network architecture
and training scheme that enables coarse-to-fine optimization with fine-grained
control over the frequency support of learned reconstructions. This is achieved
with two key innovations. First, we incorporate skip connections so that
structure at one scale is preserved when fitting finer-scale structure. Second,
we propose a novel initialization scheme to provide control over the model
frequency spectrum at each stage of optimization. We demonstrate how these
modifications enable multiscale optimization for coarse-to-fine fitting to
natural images. We then evaluate our model on synthetically generated datasets
for the the problem of single-particle cryo-EM reconstruction. We learn high
resolution multiscale structures, on par with the state-of-the art.