Universality of group convolutional neural networks based on ridgelet analysis on groups
We investigate the approximation property of group convolutional neural
networks (GCNNs) based on the ridgelet theory. We regard a group convolution as
a matrix element of a group representation, and formulate a versatile GCNN as a
nonlinear mapping between group representations, which covers typical GCNN
literatures such as a cyclic convolution on a multi-channel image,
permutation-invariant datasets (Deep Sets), and $\mathrm{E}(n)$-equivariant
convolutions. The ridgelet transform is an analysis operator of a depth-2
network, namely, it maps an arbitrary given target function $f$ to the weight
$\gamma$ of a network $S[\gamma]$ so that the network represents the function
as $S[\gamma]=f$. It has been known only for fully-connected networks, and this
study is the first to present the ridgelet transform for (G)CNNs. Since the
ridgelet transform is given as a closed-form integral operator, it provides a
constructive proof of the $cc$-universality of GCNNs. Unlike previous
universality arguments on CNNs, we do not need to convert/modify the networks
into other universal approximators such as invariant polynomials and
fully-connected networks.