OptimizationFor optimization in PS, we used the CMA-ES[17] algorithm implemented in Optuna [2] with default hyper-parameters. Specifically, we set all initial parameter values to 0.5, sigma to 1/6, and the population size to \(4+\lfloor 3\ln(n_{\text{params}})\rfloor\), where \(n_{\text{params}}\) is the number of parameters to optimize. The fitness value is defined as the accuracy for all 1069 training samples. Please note that this set is disjoint from MGSM's test set. The optimization was conducted for 1000 trials, and the best trial with respect to the training accuracy was chosen as the final model. We decided to employ TIES-Merging [49] with DARE [50] through preliminary experiments, and optimized its parameters.