@inproceedings{
ebrahimpour-boroojeny2023spectrum,
title={Spectrum Extraction and Clipping for Implicitly Linear Layers},
author={Ali Ebrahimpour-Boroojeny and Matus Telgarsky and Hari Sundaram},
booktitle={NeurIPS 2023 Workshop on Mathematics of Modern Machine Learning},
year={2023},
url={https://openreview.net/forum?id=tMCsGRtzK2}
}
We show the effectiveness of automatic differentiation in efficiently and correctly computing and controlling the spectrum of implicitly linear operators, a rich family of layer types including all standard convolutional and dense layers. we provide the first clipping method which is correct for general convolution layers, and illuminate the representational limitation that caused correctness issues in prior work.  by comparing the accuracy and performance of our methods to existing methods, using various experiments, show they lead to better generalization and adversarial robustness of the models. in addition to these advantages over the state-of-the-art methods, we show they are much faster than the alternatives.
Spectrum Extraction and Clipping for Implicitly Linear Layers
Ali Ebrahimpour-Boroojeny Matus Telgarsky Hari Sundaram