**Post-hoc altering the architecture.** We can also modify the hidden dimension \(h\) for the final layer after the model is trained. In particular, we can expand the dimensionality of \(\mathbf{W}\) by concatenating extra weight vectors that are orthogonal to the original matrix.