Although NCVX, with the PyGRANSO solver, already has many powerful features, we plan to further improve it by adding several major components: (1) symmetric rank one (SR1): SR1, another major type of quasi-Newton methods, allows less stringent step-size search and tends to help escape from saddle points faster by taking advantage of negative curvature directions [1]}; (2) stochastic algorithms: in machine learning, computing with large-scale datasets often involves finite sums with huge number of terms, calling for stochastic algorithms for reduced per-iteration cost and better scalability [2]}; (3) conic programming (CP): semidefinite programming and second-order cone programming, special cases of CP, are abundant in machine learning, e.g., kernel machines [3]}; (4) minimax optimization (MMO): MMO is an emerging modeling technique in machine learning, e.g., generative adversarial networks (GANs) [4]} and multi-agent reinforcement learning [5]}.
