Sparsification of the RNN is usually performed either at the level of individual weights (unstructured sparsification) [13, 11,<cite> 1]</cite> or at the level of neurons [14] (structured sparsification -removing weights by groups corresponding to neurons).