The multiplicative RNN (MRNN) [1]} is similar in spirit to second order tensor RNNs [2]}, [3]}, [4]}, [5]}, [6]}, [7]}. It uses a factorized hidden-to-hidden transition matrix in place of the normal RNN hidden-to-hidden matrix (\(R\)  or \(W_{zz}\) ). The MRNN can be formulated to compute an intermediate state \(\mathbf {m}_t\)  as follows:
\(\mathbf {m}_t = (W_{mx} \mathbf {x}_t) \cdot (W_{mz} \mathbf {z}_{t-1})\) 
\(\mathbf {\hat{z}}_t = W_{zm} \mathbf {m}_t + W_{zx} \mathbf {x}_t\) 
