This nuclear norm penalty is known to encourage low-rank solutions [1]}, [2]} and appears in matrix factorization problems [3]}. One can also define the gated ReLU activation, where the ReLU gates are fixed to some \(\lbrace \mathbf {h}_j\rbrace _{j=1}^m\)  [4]}
\(g({\mathbf {X}}\mathbf {w}_{1j}) := \mathrm {diag}\left(\mathbb {1}\lbrace \mathbf {X}\mathbf {h}_j \ge 0\rbrace \right) ({\mathbf {X}}\mathbf {w}_{1j}).\) 
