However, GRANSO users must derive gradients analyticallyGRANSO is implemented in MATLAB and does not support auto-differentiation, although recent versions of MATLAB have included primitive auto-differentiation functionalities. and then provide code for these computations,
a process which is often error-prone in machine learning and impractical for deep learning. Furthermore, as part of the MATLAB software ecosystem,
GRANSO is generally not compatible with popular machine/deep learning frameworks—mostly in Python and R—and users' own existing toolchains.
To overcome these issues and facilitate both high performance and ease of use in machine/deep learning,
we introduce a new software package called NCVX, whose initial release contains the solver
PyGRANSO, a PyTorch-port of GRANSO with several new and key features:
(1) auto-differentiation of all gradients, a critical feature to make PyGRANSO user-friendly;
(2) support for both CPU and GPU computations for improved hardware acceleration and massive parallelism;
(3) support for general tensor variables including vectors and matrices,
as opposed to the single vector of concatenated optimization variables that GRANSO uses;
(4) integrated support for OSQP [1]} and other QP solvers
for respectively computing search directions and the stationarity measure on each iteration.
OSQP generally outperforms commercial QP solvers in terms of scalability and speed.
All of these enhancements are crucial for solving large-scale machine/deep learning problems.
NCVX, licensed under the AGPL V3, is built entirely on freely available and widely used open-source frameworks; see https://ncvx.org for documentation and examples.
