As stated before, the models under consideration are Riemannian manifolds, therefore they can be optimized via stochastic Riemannian optimization methods such as Rsgd [1]} (we adapt the Geoopt implementation [2]}).
Given a function \(f(\theta )\)  defined over the set of embeddings (parameters) \(\theta \)  and let \(\nabla _{R}\)  denote the Riemannian gradient of \(f(\theta )\) , the parameter update according to Rsgd is of the form:
\(\theta _{t + 1} = \mathcal {R}_{\theta _{t}}(-\eta _{t} \nabla _{R}f(\theta _{t}))\) 
