Mathematical optimization is an indispensable modeling and computational tool for all science and engineering fields, especially for machine/deep learning. To date, researchers have developed numerous foolproof techniques, user-friendly solvers, and modeling languages for convex (CVX) problems, such as SDPT3 [1]}, Gurobi [2]}, Cplex [3]}, TFOCS [4]},
CVX(PY) [5]}, [6]}, AMPL [7]}, YALMIP [8]}. These developments have substantially lowered the barrier of CVX optimization for non-experts. However, practical problems, especially from machine/deep learning, are often nonconvex (NCVX), and possibly also constrained (CSTR) and nonsmooth (NSMT).
