_Remark 9.2_.: It is typically observed in practice that MoE models underperform compared to dense models with the same number of total parameters. We demonstrate that this degradation does not come from the model's knowledge storage capability.