The idea of online filters pruning via scaling factors is fundamentally similar to the concept of dropout [1]} in which units (along with their connections) are randomly dropped from the neural network during training. Considering a neural network with \(L\)  hidden layers \(\mathbf {h}_l\)  where \(l\in \lbrace 1,\cdots ,L\rbrace \) . The feed-forward operation \(f(.)\)  of this network, which is typically convolution in case of CNNs, with dropout is represented as:
\(\mathbf {h}_l = f(\mathbf {h}_{l-1}\odot \mathbf {z}_{l-1}).\) 
