@inproceedings{
kaba2023symmetry,
title={Symmetry Breaking and Equivariant Neural  Networks},
author={S{\'e}kou-Oumar Kaba and Siamak Ravanbakhsh},
booktitle={NeurIPS 2023 Workshop on Symmetry and Geometry in Neural Representations},
year={2023},
url={https://openreview.net/forum?id=d55JaRL9wh}
}
Using symmetry as an inductive bias in deep learning has been proven to be a principled approach for sample-efficient model design. However, the relationship between symmetry and the imperative for equivariance in neural networks is not always obvious. Here, we analyze a key limitation that arises in equivariant functions: their incapacity to break symmetry at the level of individual data samples. In response, we introduce a novel notion of 'relaxed equivariance' that circumvents this limitation. We further demonstrate how to incorporate this relaxation into equivariant multilayer perceptrons (E-MLPs), offering an alternative to the noise-injection method. The relevance of symmetry breaking is then discussed in various application domains: physics, graph representation learning, combinatorial optimization and equivariant decoding.
Symmetry Breaking and Equivariant Neural  Networks
Sékou-Oumar Kaba Siamak Ravanbakhsh