@inproceedings{
granberry2023soequivariant,
title={{SO}(3)-Equivariant Representation Learning in 2D Images},
author={Darnell Granberry and Alireza Nasiri and Jiayi Shou and Alex J Noble and Tristan Bepler},
booktitle={NeurIPS 2023 Workshop on Symmetry and Geometry in Neural Representations},
year={2023},
url={https://openreview.net/forum?id=CeG8jzTL2k}
}
Imaging physical objects that are free to rotate and translate in 3D is challenging. While
an object’s pose and location do not change its nature, varying them presents problems
for current vision models. Equivariant models account for these nuisance transformations,
but current architectures only model either 2D transformations of 2D signals or 3D trans-
formations of 3D signals. Here, we propose a novel convolutional layer consisting of 2D
projections of 3D filters that models 3D equivariances of 2D signals—critical for capturing
the full space of spatial transformations of objects in imaging domains such as cryo-EM. We
additionally present methods for aggregating our rotation-specific outputs. We demonstrate 
improvement on several tasks, including particle picking and pose estimation.
SO(3)-Equivariant Representation Learning in 2D Images
Darnell Granberry Alireza Nasiri Jiayi Shou Alex J Noble Tristan Bepler