NotationsTables 4 to 7 present the results of deactivating certain neurons, where "Under" denotes the understanding layers, "S-ATTN" and "S-FFN" correspond to the self-attention and the feed-forward within the task-solving layers respectively, "Gen" refers to the generation layers. The term "Random" is used to describe deactivating randomly chosen neurons, whereas "Lang-Spec" refers to the deactivation of language-specific neurons. We also present the gap between the original performance and performance after deactivation for English (\(\Delta_{\text{Eng}}\)) and averaged non-English languages (\(\Delta_{\text{n-Eng}}\)), respectively. A single metric \(\Delta\) is then introduced as \(\Delta_{\text{Eng}}-\Delta_{\text{n-Eng}}\), where a high value indicates such deactivation operation does not bring much impact to the English performance but lead to performance drop in non-English.