We report the data statistics for the CKL benchmark in Table 5. We measure the size, average input token length, average answer token length, and the answer types of each constructed dataset. One thing to consider is that LAMA (Petroni et al., 2019) from which we constructed InvariantLAMA is originally constructed for only single-token decoding (1.3 with the T5-tokenizer) because multi-token decoding entails additional, tunable parameters (beam size, n-gram repetition penalties, etc.).