Evaluation ConfigurationsFor T5 based models, all evaluation is done in a zero-shot manner and is processed with a single GPU. For InvariantLAMA, the input and output length is fixed as 25 and 4 respectively. For UpdatedLAMA and NewLAMA, the input and output length is 50 and 10 respectively. Lastly, the input and output length is 150 and 10 respectively for NewLAMA-Easy. The rationale of this hyperparameter is based on average input and answer token in Table 5.