Furthermore, we find that deactivating the \(k\)-th neuron of \(W_{down}\) is equivalent to deactivating the \(k\)-th neuron in \(W_{up}\) as they all set \(h_{\text{ffn}}[k]=0\). Therefore \(\text{Imp}(W_{down}|x)\) can be obtain by Equation (5).