To investigate _why_ the LLaMA architecture is inferior to GPT2 in the 100-exposure (insufficiently trained) setting, we closely examine LLaMA by gradually modifying its architecture _back towards_ GPT2 to identify the key architectural changes. We start by tying weights, as this enhances tiny LLaMA model's capacity in the 1000-exposure setting (Result 5).