We further introduce a non-linear activation formulation for LRBP, following [1]}, who argued that non-linearity both before and after the Hadamard product complicates the gradient computation. This addition to Equation (REF ) can be represented as follows:
\(\begin{array}{rcl}\mathbf {f} &=& \mathbf {P}^{T}\tanh (\mathbf {U}^{T}\mathbf {x} \circ \mathbf {V}^{T}\mathbf {y}) + \mathbf {b}\end{array}\) 
