We implement the idea for LSTM in two frameworks: pruning [14] and Bayesian sparsification <cite>[1]</cite> and observe that resulting gate structures (which gates are constant and which are not) vary for different NLP tasks.