Differentially private Riemannian optimization
In this paper, we study the differentially private empirical risk
minimization problem where the parameter is constrained to a Riemannian
manifold. We introduce a framework of differentially private Riemannian
optimization by adding noise to the Riemannian gradient on the tangent space.
The noise follows a Gaussian distribution intrinsically defined with respect to
the Riemannian metric. We adapt the Gaussian mechanism from the Euclidean space
to the tangent space compatible to such generalized Gaussian distribution. We
show that this strategy presents a simple analysis as compared to directly
adding noise on the manifold. We further show privacy guarantees of the
proposed differentially private Riemannian (stochastic) gradient descent using
an extension of the moments accountant technique. Additionally, we prove
utility guarantees under geodesic (strongly) convex, general nonconvex
objectives as well as under the Riemannian Polyak-{\L}ojasiewicz condition. We
show the efficacy of the proposed framework in several applications.