Due to the limited space, we employ a more concise notation. We denote deactivating neurons in the self-attention layer of the \(i\)-th layer as \(D_{i}^{(A)}\), while deactivating neurons in the feed-forward layer of the \(i\)-th layer is denoted as \(D_{i}^{(F)}\). We denote \(\mathcal{U}=\{1,\cdots,N_{1}\}\) as the set of layers that take charge of understanding as shown in Figure 2. Similarly, we denote \(\mathcal{S}=\{N_{1}+1,\cdots,N_{2}\}\) as the set of layers that take charge of task solving and \(\mathcal{G}=\{N_{2}+1,\cdots,32\}\) as the set of layers that take charge of generation5. Furthermore, \(D_{\mathcal{U}}^{(A)}\) represents deactivating neurons in self-attention layers of \(\mathcal{U}\). Similarly, we introduce \(D_{\mathcal{U}}^{(F)}\), \(D_{\mathcal{S}}^{(A)}\), \(D_{\mathcal{S}}^{(F)}\), \(D_{\mathcal{G}}^{(A)}\) and \(D_{\mathcal{G}}^{(A)}\).