In this paper, we present Canonical Field Network (CaFi-Net), a self-supervised method for category-level canonicalization of the 3D position and orientation of objects represented as neural fields, specifically neural radiance fields.
Canonicalizing neural fields is challenging because,
unlike 3D point clouds or meshes, neural fields are continuous, noisy, and hard to manipulate since they are parameterized as the weights of a neural network [1]}.
To address these challenges, we first extend the notion of equivariance to continuous vector fields and show how networks for processing 3D point clouds [2]} can be extended to operate directly on neural radiance fields.
We design CaFi-Net as a Siamese network
that contains layers to extract equivariant features directly on vector fields.
These field features are used to learn a canonical frame that is consistent across instances in the category.
During inference, our method takes as input neural radiance fields of object instances from a category at arbitrary pose, and estimates a canonical field that is consistent across the category.
To handle noise in radiance fields from NeRF, our method incorporates density-based feature weighting and foreground-background clustering.
