Here, we seek to provide some additional intuition around the constrained nuclear norm \(\Vert \mathbf {Z}\Vert _{*, \mathrm {K}}\)  for a simple case, to contrast the regularization for convex ReLU against convex linear and gated ReLU networks. We provide a visualization for \(\Vert \mathbf {Z}\Vert _{*, \mathrm {K}}\)  compared to \(\Vert \mathbf {Z}\Vert _*\)  in the case that \(\mathbf {K} = \mathbf {I}\) . This would occur in a ReLU network when \(\mathbf {X} = \mathbf {I}\)  and we encounter a particular hyperplane arrangement \(\mathbf {D}_j = \mathbf {I}\) . One can also note that in ReLU MLPs where \(n \le d\)  and the data \({\mathbf {X}}\)  is whitened, the convex optimization objective will reduce to a linear model with this norm as its regularization [1]}.
