In this section, we introduce the preliminary chain-of-thought reasoning with LLMs, and we refer to the formula definition in Qiao et al. (2023). Suppose there is a question \(\mathcal{Q}\), a prompt \(\mathcal{T}\) and a probabilistic language model \(P_{LM}\). The model takes the question and prompt as inputs to give the rationale \(\mathcal{R}\) and answer \(\mathcal{A}\). We first consider in-context scenarios where the demonstrations do not contain reasoning chains. We need to maximize the likelihood of Answer \(\mathcal{A}\), as shown in Equ (1,2).