RECAPP: Crafting a More Efficient Catalyst for Convex Optimization
The accelerated proximal point algorithm (APPA), also known as "Catalyst", is
a well-established reduction from convex optimization to approximate proximal
point computation (i.e., regularized minimization). This reduction is
conceptually elegant and yields strong convergence rate guarantees. However,
these rates feature an extraneous logarithmic term arising from the need to
compute each proximal point to high accuracy. In this work, we propose a novel
Relaxed Error Criterion for Accelerated Proximal Point (RECAPP) that eliminates
the need for high accuracy subproblem solutions. We apply RECAPP to two
canonical problems: finite-sum and max-structured minimization. For finite-sum
problems, we match the best known complexity, previously obtained by
carefully-designed problem-specific algorithms. For minimizing $\max_y f(x,y)$
where $f$ is convex in $x$ and strongly-concave in $y$, we improve on the best
known (Catalyst-based) bound by a logarithmic factor.