Since in text classification tasks, usually only a small number of input words are important, we use additional multiplicative weights to sparsify the input vocabulary following Chirkova et al. <cite>[1]</cite> .