Stochastic Frank Wolfe methods for nonconvex optimization
We study Frank Wolfe methods for nonconvex stochastic and finite sum optimization problems. Frank Wolfe methods (in the convex case) have gained tremendous recent interest in machine learning and optimization due to their projection free property and their ability to exploit structured constraints. However, our understanding of these algorithms in the nonconvex setting is fairly limited. In this paper, we propose nonconvex stochastic Frank Wolfe methods and analyze their convergence properties. Furthermore, for objective functions that decompose into a finite sum, we leverage ideas from variance reduction for convex optimization to obtain new variance reduced nonconvex Frank Wolfe methods that have provably faster convergence than the classical Frank Wolfe method.
2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton)