It is worth noting that although the questions are based on information the models were not exposed to during training, the results of the base models surpass \(\frac{1}{L}=0.25\). This can partially be explained by the models using reasoning and/or pre-existing knowledge when answering questions that are not independent of the past information. Some examples of this can be found in Appendix C.