Such two-point zeroth-order gradient estimators have frequently been studied in zeroth-order optimization works (see e.g. [1]}). To facilitate efficient escape from saddle points, our proposed algorithm:ZOPGD adds isotropic perturbation at each iteration.
