We have selected a broad range of GPT2-\(\ell\)-\(h\) models with practical \(\ell\) and \(h\) values, excluding those with similar model sizes. Their selection is detailed in Figure 1, encompassing both wide and shallow transformers (e.g., GPT2-2-20, GPT2-3-20, GPT2-4-20) and skinny and deep transformers (e.g., GPT2-16-4, GPT2-16-8, GPT2-28-20). For reference, GPT2 small/med/large correspond to GPT2-12-12, GPT2-24-16, GPT2-36-20, respectively.