where \(\widetilde{\ell }_{\lambda ,\mu }(\psi ) = \ell (\psi ) + \sum _{g=1}^3 \sum _{j=1}^{d_g}{\widetilde{p}}_{\lambda _1}(|\beta _{gj}|) + \sum _{g\ne g^{\prime }}\sum _{j=1}^{d_g} {\widetilde{p}}_{\lambda _2}(|\beta _{gj}-\beta _{g^{\prime }j}|) + {\widetilde{\Omega }}_{\lambda _2,\mu }(\psi ).\)  Towards optimizing (REF ), define the vector-valued soft thresholding operation \(\mathbf { S}_{\lambda }(\mathbf { x})\)  at the \(j\) th element by \([\mathbf { S}_{\lambda }(\mathbf { x})]_j = \operatorname{sign}(x_j)\max \lbrace  0, |x_j| - \lambda \rbrace \) . Then the \(m\) th step of the iterative proximal gradient algorithm is \(\psi ^{(m)} \leftarrow \mathbf { S}_{\lambda _1}\left( \psi ^{(m-1)} - r^{(m)} \cdot \nabla \widetilde{\ell }_{\lambda ,\mu }(\psi ^{(m-1)}) \right)\) , where \(r^{(m)}\)  is an adaptive step size determined by backtracking line search [1]}. Iterations continue until one or more stopping criteria fall below some threshold \(\xi >0\) , such as change in objective function \(|Q_{\lambda ,\mu }(\psi ^{(m)}) - Q_{\lambda ,\mu }(\psi ^{(m-1)})| < \xi \) , or norm of the change in estimates \(\Vert \psi ^{(m)} -\psi ^{(m-1)}\Vert  < \xi \) .
